http://orcid.org/0000-0002-3224-8858, Thomas Lin Pedersen, and Panagiotis Skintzos.
by Paul Murrellhttp://orcid.org/0000-0002-3224-8858, Thomas Lin Pedersen, and Panagiotis Skintzos.
Version 1: Monday 22 May 2023
This document
is licensed under a Creative
Commons Attribution 4.0 International License.
This document describes an upate to the behaviour of Porter-Duff compositing operators in R graphics.
R graphics follows a "painter's model" where later drawing occurs "on top of" earlier drawing. For example, the following code defines two rectangles, one blue and one red.
library(grid)
src <- rectGrob(x=2/3, y=1/3, width=.5, height=.5, gp=gpar(col=NA, fill=rgb(0, 0, .9))) dst <- rectGrob(x=1/3, y=2/3, width=.5, height=.5, gp=gpar(col=NA, fill=rgb(.7, 0, 0)))
If we just draw these rectangles one at a time, first the red and then the blue, the blue rectangle is drawn on top of the red rectangle where they overlap. In terms of Porter-Duff compositing operators (Porter and Duff, 1984), this "painter's model" corresponds to using an "over" operator to combine the rectangles.
grid.draw(dst) grid.draw(src)
R version 4.2.0 introduced the ability to explicitly set the
compositing operator for drawing (Murrell, 2021).
The 'grid' function grid.group() allows us
to specify a "source" object, a "destination" object,
and a compositing operator.
The default drawing behaviour in R
corresponds to drawing a group and using an "over" compositing operator,
as shown by the following code.
grid.group(src, "over", dst)
grid.group(src, "out", dst)
The operators that we can choose from include the full set of Porter-Duff operators:
"clear" "source" "over" "in" "out" "atop" "dest" "dest.over" "dest.in" "dest.out" "dest.atop" "xor"
Support for these operators was only added to graphics devices that were based on the Cairo Graphics library initially (Packard et al., 2021), under the assumption that that implementation was "standard". However, an attempt to add support to the graphics device provided by the 'ggiraph' package (Gohel and Skintzos, 2023) highlighted some "non-standard" behaviour in the Cairo-based devices. In particular, the "clear" operator and "src" operators, in R 4.2.0, produce the results shown below.
However, the original Porter-Duff definition suggests that the behaviour should look like the results below. This is now the behaviour on the Cairo-based devices in R 4.3.0.
For completeness, the
correct behaviour for graphics devices that support Porter-Duff
compositing operators should now conform
to the image below. The Cairo devices and the quartz()
device currently support compositing operators and adhere to
this behaviour. Support is under development for
the graphics devices provided by the packages
'ggiraph', 'svglite' (Wickham et al., 2023), and
'ragg' (Pedersen and Shemanarev, 2023).
The remainder of this document surveys the definition of Porter-Duff operators from a variety of sources in order to explain/justify the choice of the behaviour that is outlined above for R graphics devices.
The canonical definition of the Porter-Duff operators is the original article from 1984. The table below is taken from that article1.
This table actually describes the results for a single image pixel, with $A$ and $B$ representing a source and a destination object that partially overlap the pixel, dividing it into four regions: a region that is overlapped by neither $A$ nor $B$, a region that is only overlapped by $A$, a region that is only overlapped by $B$, and a region that is overlapped by both $A$ and $B$. The "quadruple" column defines, for each compositing operator, what the output should be for each of the four regions. For example, the "clear" operator (the first row) says that all four regions should end up empty, whereas the "xor" operator (the last row) says that the region of no overlap should be empty, the region with only $A$ overlap should produce $A$, the region with only $B$ overlap should produce $B$, and the region with both overlap should be empty.
The output for a pixel that is partially overlapped by both $A$ and $B$ requires a calculation involving the amount of coverage of $A$ and $B$ (represented by $\alpha_A$ and $\alpha_B$), but if we just consider the simpler case of pixels that are either entirely covered or not covered at all ($\alpha_A$ and $\alpha_B$ are both either 0 or 1) then the "quadruple" column describes what the output should be for pixels that are covered by neither $A$ nor $B$, pixels that are only covered by $A$, pixels that are only covered by $B$, and pixels that are covered by both $A$ and $B$. This means that the "diagram" column shows not only how $A$ and $B$ contribute to the output for a single pixel, but how $A$ and $B$ contribute to the image overall (for each of the compositing operators). For example, the "clear" operator says that the output should be empty for the entire image, whereas the "xor" operator says that the output should be empty where pixels are covered by neither $A$ nor $B$, the output should be $A$ where pixels are only covered by $A$, the output should be $B$ where pixels are only covered by $B$, and the output should be empty where pixels are covered by both $A$ and $B$.
Pedersen, 2013 provides a similar explanation, with more detail and helpful diagrams.
The W3C Compositing and Blending Candidate Recommendation (Cabanier and Andronikos, 2015), which is referenced by the W3C Scalable Vector Graphics (SVG) 2 Candidate Recommendation (Bellamy-Royds et al., 2018), proposes the same behaviour. The images in the table below are taken from the W3C Candidate Recommendation2; in this table, the "copy" corresponds to the "source" operator. Although these may only be "aspirational" images, given the source, they seem like a reasonable indication of what might be the correct result.
| op | op | op | op | ||||
|---|---|---|---|---|---|---|---|
| src over |
|
clear |
|
copy |
|
src in |
|
| src out |
|
src atop |
|
dst |
|
dst over |
|
| dst in |
|
dst out |
|
dst atop |
|
xor |
|
The W3C Scalable Vector Graphics (SVG) 1.1 Recommendation (Dahlström et al., 2011) is the current official description of SVG. This allows compositing operators via filters. Quite a range of results is possible, but the possibilities include a set of results that are consistent with the behaviour proposed in this document.
The code in svg.R generates a set of SVG files that are shown in the table below. The blue rectangle is "source" and the red rectangle is "destination". The images below are PNG versions (generated using Inkscape) to avoid any issues with browser incompatibilities. Only some of the operators are shown because the SVG filters allow "source" and "destination" to be swapped and the SVG filters do not support the "clear" operator. For example, only "src in" is shown, but we could get "dst in" by making the blue rectangle the "destination" and the red rectangle the "source".
| op | op | op | op | op | |||||
|---|---|---|---|---|---|---|---|---|---|
| over |
|
in |
|
out |
|
atop |
|
xor |
|
There appears to be an interesting history behind the development of compositing operator support in SVG, which has lead to the wide range of possibilities. Importantly, the following quote refers to "true" Porter-Duff implementations "blast"ing the entire canvas for the "clear" operator, which is part of the behaviour proposed in this document, although it appears to have been used as a reason for not including "clear" support in SVG.
The group was (understandably) concerned that the
introduction of some of the Porter-Duff operators could
affect pixels outside of the geometry of the object and
that would not be possible to implement on top of existing
rendering libraries. Notably ASV3 just sits on top of the
standard core libraries used in other products, whilst Batik
sits on top of Java2D. True Porter-Duff implementations
can blast the entire canvas if you do something like 'clear'
and so this was considered to be a bad precedent.
As mentioned in the Introduction, the (naive) behaviour of
Porter-Duff compositing operators in the Cairo Graphics library
(just using cairo_set_operator())
differs from the interpretation that is proposed in this
document. The Cairo
Graphics documentation provides a detailed description
of the intended behaviour, and includes diagrams showing that
the "clear" and "source" operators leave the "destination"
object intact where it is not overlapped by the "source"
(as shown in the image in the Introduction).
The Cairo Graphics documentation describes this as a "bounded" interpretation of these operators. Unfortunately, as shown in subsequent sections, this interpretation is not consistent with other graphics systems, so would be difficult to implement across graphics devices. Furthermore, we believe it is not consistent with the original Porter-Duff definitions, which we have (so far) been able to implement across a variety of graphics devices.
Behaviour that is consistent with the original Porter-Duff
definitions is possible using the Cairo Graphics library by
adding a cairo_push_group() and
cairo_pop_group() around each shape that is drawn
when the compositing operator is "clear" or "source".
Quartz 2D is a graphics system for MacOS computers
(Apple Inc., 2017).
The (naive) implementation of Porter-Duff operators using
Quartz 2D (just using CGContextSetBlendMode())
also differs from the interpretation that is proposed in this
document. The table below shows the result, which is similar
to the Cairo graphics behaviour ("clear" and "source" retain
the "destination"), but has additional variations of its own:
the "in", "out", "dest.in", and "dest.atop" operators
also retain the "destination".
Behaviour that is consistent with the original Porter-Duff
definitions is possible using Quartz 2D by
drawing each shape (that is drawn
when the compositing operator is "clear", "source", "in", "out",
"dest.in", or "dest.atop")
on its own layer (with CGLayerCreateWithContext()).
Java 2D is a graphics system for the Java Programming Language (Oracle, 2018). We have not tested the Java 2D behaviour directly, but the Oracle Java2D Compositing Tutorial contains diagrams that mostly show consistency with the original Porter-Duff definitions. However, the "clear" operator is an exception, with behaviour that is consistent with the Cairo Graphics library (and not the Porter-Duff interpretation proposed in this document). The Java2D documentation for compositing operators contains descriptions that also sound consistent with the Cairo Graphics behaviour for the "source" operator, but the Tutorial does not provide diagrams to confirm that.
This document sets out the defined behaviour of Porter-Duff operators for R graphics devices. The proposed behaviour is consistent with the original Porter-Duff definitions and with W3C Candidate Recommendations and it can be implemented using the W3C SVG Recommendation, using the Cairo Graphics library, and using Quartz 2D (on MacOS).
The proposed behaviour is not consistent with Java2D, nor a naive implementation in either Cairo Graphics or Quartz 2D. However, none of those three behaviours is fully consistent with each other nor, we would argue, are they consistent with the original Porter-Duff definitions.
One final point to note is that the current R graphics interface conflates compositing operators with blend modes, which are conceptually separate stages in the rendering process (see, for example, Porter/Duff Compositing and Blend Modes by Søren Sandmann Pedersen, the W3C's Compositing and Blending Level 1, and the historical Comments on SVG Compositing). In the current R graphics interface, selecting a compositing operator implies a "normal" blend mode, while selecting a blend mode implies an "over" compositing operator. Future work may look at separating these concepts to allow other combinations of compositing operators with blend modes, although support for all possible combinations may be difficult to achieve on many graphics devices.
The examples and discussion in this report relate to R version 4.3.0.
This report was generated within a Docker container (see Resources section below).
Murrell, P., Pedersen, T. L., and Skintzos, P. (2023). "Porter-Duff Compositing Operators in R Graphics" Technical Report 2023-02, Department of Statistics, The University of Auckland. Version 1. [ bib | DOI | http ]
The refactoring of the code for the Cairo devices (to correct the behaviour of Porter-Duff compositing operators) accidentally fixed a bug in the drawing of shapes that have both a (stroked) border and a (filled) interior when a mask is in force. This section briefly outlines the bug and explains the (accidental) fix.
The issue revolves around the fact that drawing a shape that has both a border and an interior (e.g., a circle, but not a line) actually involves two drawing operations: one to fill the interior and one to stroke the border. This is not obvious when drawing with opaque colours and/or thin borders, but becomes obvious with semitransparent colours and thick borders. For example, the following code draws a circle with a thick opaque blue border and an opaque red fill. The fill is drawn first and then the border is drawn on top. What we cannot see is that the blue border partially overlaps the red fill (because the border is drawn along the circumference of the circle and a thick border extends both inside and outside the circumference of the circle).
grid.circle(r=.3, gp=gpar(col=rgb(0,0,1), fill=rgb(1,0,0), lwd=20))
This becomes obvious with the following code, which draws a circle with a thick semitransparent blue border and a semitransparent red fill. The border is centred on the circumference of the circle, extending both outside and inside the circumference, and the part of the semitransparent blue border that extends inside the circumference overlaps with the semitransparent red fill (producing a purple region).
grid.circle(r=.3, gp=gpar(col=rgb(0,0,1,.5), fill=rgb(1,0,0,.5), lwd=20))
Now consider the following code, which draws the first circle (opaque border and opaque fill) with a semitransparent mask in force. This is the correct result: the fill is drawn with the mask in force, so the fill becomes semitransparent, and then the border is drawn with the mask in force, so the border also becomes semitransparent (and we can see the border overlapping the fill).
pushViewport(viewport(mask=rectGrob(gp=gpar(fill=rgb(0,0,0,.5))))) grid.circle(r=.3, gp=gpar(col=rgb(0,0,1), fill=rgb(1,0,0), lwd=20))
However, in R versions 4.2.*, we would actually get the result below. The fill is semitransparent and the border is semitransparent, but there is no overlap between border and fill. This is the bug that was accidentally fixed.
The result above is arguably the more useful result, but it is inconsistent with the normal R graphics behaviour (stroke and fill are separate drawing operations). Fortunately, we can get the result above (on purpose) if we draw the circle (opaque border and fill) as a group (with the mask in force). This code draws the (opaque) fill then the (opaque) border as a separate image (that looks like the first image in this section) and then adds that image to the main image with the mask in force, which turns the whole (separate) image semitransparent.
pushViewport(viewport(mask=rectGrob(gp=gpar(fill=rgb(0,0,0,.5))))) grid.group(circleGrob(r=.3, gp=gpar(col=rgb(0,0,1), fill=rgb(1,0,0), lwd=20)))
1 The copyright notice at the bottom of the front page of the article states that "Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery." Consider this notice given.
2 Copyright © 2014 World Wide Web Consortium. https://www.w3.org/Consortium/Legal/2023/doc-license.
This document
by Paul
Murrell is licensed under a Creative
Commons Attribution 4.0 International License.