We're probably going to need to explain the parameters in a bit more depth because their behavior is non-idiomatic for WebGPU and different from the analogous parameters to copyExternalImageToTexture()...

It's important to call out the fact that sx, sy, swidth, and sheight are specified in CSS pixels, and that they specify clipping behavior rather than scaling. This is almost certainly the only WebGPU function with parameters in CSS pixels.

The width and height parameters are in texel coordinates, and they specify scaling behavior, which is different from the analogous parameters to copyExternalImageToTexture(). As a general rule, WebGPU functions do not support implicit scaling (this is in contrast to WebGL, which does), so this behavior is non-idiomatic and should be clearly stated.

The justification for permitting implicit scaling is that the scaling can be done by the web content rendering implementation using the same method employed for regular web content. That will ensure consistency of results when the same web content is rendered inside our outside of a canvas. The core WebGPU implementation will not do any implicit scaling, which is in keeping with WebGPU principles.

Indeed, that was confusing, see my other comment ^^. In any case we should most likely put it in the source dict.

Ok, so if I understand correctly, the methods take the CSS coordinate rect of the element specified by sx, sy, swidth, sheight, rasterize it as an image of size width, height and copy it into destination.texture at offset destination.origin?

sx, sy, swidth, sheight obviously belong in the source dict, maybe as an optional GPUCssRect (origin: GPUCssOrigin2D = {0, 0}, optional size: GPUCssExtent2D with size defaulting to the full size of the element after the origin?). What's difficult to find a spot for is width/height as it's a copy size but also a scale.

Maybe this function should be blitElementImageToTexture in which case it's clearly a rect to rect copy, with the width/height specified in the destination dict?

@kainino0x WDYT?

For a resizing "blit", I would put a CSS width,height in the source dictionary and a texture width,height in the destination dictionary. You can make a new dictionary that extends GPUCopyExternalImageDestInfo with an extra field.

copyExternalImageToTexture is a blit, so it seemed intuitive to me that the new API would also be a blit of the rendered web content. width and height are sort of doing double duty here: they define a scaling factor for web content rendering, but conceptually they also define clipping boundaries for the blit.

Note that in copyExternalImageToTexture, the origin parameter provides an offset into the source image. For the new API, that would be redundant with sx/sy, so there is no plan for an origin parameter.

copyExternalImageToTexture doesn't allow resizing. It has just one copySize which is both the source rectangle size and the destination rectangle size. That is an option for HIC too, but I do think HIC should allow resizing because it's not coming from a raster source that inherently has discrete pixels.

(In theory copyExternalImageToTexture actually probably should have a way to control resizing, at least when using an SVG source, because the image can actually be rasterized at the target resolution rather than "resized".)

Note that in copyExternalImageToTexture, the origin parameter provides an offset into the source image. For the new API, that would be redundant with sx/sy, so there is no plan for an origin parameter.

FWIW there's an argument that sx/sy should be called origin for consistency with other places in WebGPU but I personally don't really care since sx/sy are consistent with other HIC/HTML things.

These multiple variants to let some parameters be optional seem very WebGL-like. In WebGPU the closest equivalent to copyElementImageToTexture is copyExternalImageToTexture where the parameters for the rectangle to copy from/to are:

• for the source: source.origin and size copySize.
• for the destination: destination.origin and size copySize.

So it seems a single overload is necessary:

typedef (Element or ElementImage) GPUCopyImageElementSource;

dictionary (Element or ElementImage) {
    required GPUCopyImageElement source;
    GPUOrigin2D origin = {};
    boolean flipY = false;
};

    undefined copyElementImageToTexture(
        GPUCopyElementImageSourceInfo source,
        GPUCopyExternalImageDestInfo destination, // type likely renamed
        GPUExtent3D copySize);

Or am I missing what these parameters try to do? (is this where we choose how big an element should be rasterized as by the browser? That'd be in the (Element or ElementImage)).

I will take a stab at updating the parameters to be more WebGPU-y.

@Kangz -- are there WebGPU typedefs for floating-point offset and size? The closest analog I could find is GPURenderPassEncoder, which just uses individual floats. Do you think we should define GPUCssOffset/GPUCssSize types so that they're somewhat self-documenting, or is that overkill?

If it's just typedefs it doesn't change the API, so I wouldn't worry about until we're landing the final spec. double (or whatever) is fine.

Here's a sketch of what I think it would look like. This has separate src/dst sizes.

dictionary GPUCopyElementImageSourceInfo {
    required (Element or ElementImage) source;
    double x = 0;
    double y = 0;
    double width; // optional, defaults to (natural width - x)
    double height; // optional, ^
    boolean flipY = false; // unsure if we need this
};

dictionary GPUCopyElementImageDestInfo
         : GPUCopyExternalImageDestInfo {
    GPUIntegerCoordinate width; // optional, defaults to (approx) src.width *
                                // (srcParentCanvas.width / srcParentCanvas.clientWidth),
                                // clamped to fit inside the texture
    GPUIntegerCoordinate height; // optional, ^
};

partial interface GPUQueue {
    undefined copyElementImageToTexture(
        GPUCopyElementImageSourceInfo source,
        GPUCopyElementImageDestInfo destination);
};

There's some possible refinement like grouping together x/y or width/height or even all four for a "rect" type. However that's less important to work out right now, what's important is that both source and destination have their own rects, and what the defaulting behavior is

nit: Avoid the name GPUImageCopyTextureTagged, that is the old name of GPUCopyExternalImageDestInfo and we haven't gotten around to updating it in Blink because it's not web-exposed.

Just so I'm clear, this means that by default, the rasterized element image we receive is at 1 CSS px = 1 WebGPU texel? If yes, sounds good.

No, that's not correct.

Just to make sure we're clear on terminology: when you say "by default", I interpret that as "neither width nor height is specified". Since sx/sy/swidth/sheight don't affect scaling they aren't relevant to this point.

By default, the rasterized element image will be scaled by the inverse of the canvas-grid-to-CSS scaling factor of the <canvas> element containing the drawn element. To make this concrete, the following two invocations should behave identically (modulo numerical precision differences due to javascript using double-precision floats):

const canvas = document.querySelector('canvas');
const element = canvas.firstElementChild;

queue.copyElementImageToTexture(element, texture, {});

queue.copyElementImageToTexture(
    element, texture,
    { width: getComputedStyle(element).width * (canvas.width / canvas.clientWidth),
      height: getComputedStyle(element).height * (canvas.height / canvas.clientHeight) });

Note that getComputedStyle(element).width and canvas.clientWidth are in CSS pixels, and canvas.width defines the pixel grid of the canvas' backing store.

There is a lot of subtle complexity around this topic, which was previously plumbed at depth in a spec issue. The clearest summary may be this comment, but to restate it here:

The overriding principal is that by default, if an element image is copied to a texture and that texture is blitted to the canvas containing the element, the on-screen size and proportion of the element will match what it would be if the element were placed outside the canvas.

I see, thank you for the explanation. Apologies if this is covered in the long thread but I couldn't find it by searching a few keywords:

How would a developer usefully use the case where width/height aren't specified ({} above)? They would need to know exactly what size to make the GPUTexture beforehand. It's not good enough to know approximately, or know an upper bound, they need to know exactly, otherwise the copy might fail or they might have a 1px gap along the edge of their texture.

From what I understand of sizing, the only way to get the exact number would be through ResizeObserver which is (1) annoying, and (2) doesn't make that much sense here anyway since the element is not actually directly visible on the page.

So my proposal would be, at least for the first iteration, that the WebGPU (and likely WebGL) version of HIC doesn't have a default sizing at all, it just makes the developer specify the size they want. I think that in almost all cases developers will just want to set it to texture.width, texture.height (however they determined those - perhaps with a fixed ratio of CSS px to WebGPU texel, or with getComputedStyle or ResizeObserver).

@kainino0x -- the problem with requiring a size specification is that it can cause fuzziness if the intrinsic size of the element is fractional. For example, if the intrinsic size of an element is (99.6, 100.4) and the API is invoked with explicit destination size of (100, 100), then the implementation will apply a tiny scaling factor when rasterizing the web content. We considered a few ways to address this:

1. Make destination size parameters floating-point. Destination size is in pixel grid coordinates, and it's pretty terrible to specify pixel grid dimensions using floating-point numbers. This feels developer-unfriendly and destined to create confusion. This is also not guaranteed to fix the fuzziness problem: if the destination size is computed using floating-point arithmetic with JavaScript doubles, it may produce a different result than the same arithmetic done using our internal single-precision floats.
2. If the specific destination size is equal to round(intrinsic_size), then use the intrinsic size for scaling. This is subtly magical behavior and also likely to create confusion.
3. Use separate parameters to specify scaling vs. clipping. So there would be a source rect to specify clipping in CSS space; a destination size to specify scaling; and a destination clip rect to specify what area of the texture to modify. That just seems like a god-awful mess of somewhat redundant parameters.

In practice, I expect developers to size their textures based on the content, for example:

// expand the source rect by this number of CSS pixels to capture ink overflow
const outset = 50;
const boxWidth = Number.parseFloat(getComputedStyle(element).width);
const boxHeight = Number.parseFloat(getComputedStyle(element).height);
const canvasScaleFactorX = canvas.width / canvas.clientWidth;
const canvasScaleFactorY = canvas.height / canvas.clientHeight;
const textureWidth = Math.ceil(canvasScaleFactor * (boxWidth + 2*outset));
const textureHeight = Math.ceil(canvasScaleFactor * (boxHeight + 2*outset));
const texture = device.createTexture({ size: [ texture width, textureHeight ]  });

// No destination rect is specified; intrinsic sizing is used and the rasterized
// output will be clipped to the size of the texture.
queue.copyElementImageToTexture(
  element,
  { x: -outset, width: boxWidth + 2*outset,
    y: -outset, height: boxHeight + 2*outset },
  texture);

Using Math.ceil introduces the possibility of a one-pixel edge gap, but that is unavoidable with this API. The developer may choose to use Math.floor instead, in which case the rasterized content may be clipped by one pixel at the edges -- also unavoidable with this API. Pick your poison! But we think it's important to support the un-scaled intrinsic-size behavior to prevent fuzziness in the output, and the developers may choose how they want to snap the texture size.

Ah, I didn't think about making it clip instead of fail. Normally in WebGPU if you try to specify a destination rectangle that goes outside the destination, that would be an error. But this isn't really a "copy" in the same sense, so it doesn't seem like too big a deal for it to be different.

I agree those three options don't sound nice. (Just so we're on the same page I am assuming the suggested change where there's both a source width/height and a destination width/height.)

I'm probably willing to accept it with the clipping as it avoids device-specific failures, but with the following notes:

• It doesn't seem inherent to the design of the API that the image would be fuzzy if resized. In theory couldn't the browser re-rasterize the content at the exact resolution of the copy, same as it would do with a CSS transform: scale()? I actually feel that should be the design intent of the API we standardize, even if it allows for post-raster resizing - and should be phrased as such (e.g. "render" or "draw" rather than "copy" or "blit").
• It's very easy for developers to miss the ceil and have bugs on some devices (where the content size rounds up). If copyElementImageToTexture can compute an exact integer size, can we just expose that rather than making people compute it themselves? It would at least be less error prone.

Fair, sorry it's very easy to bikeshed ^_^
I don't think the name will matter that much to developers as long as it's not too ambiguous. I would say match either 2d canvas or WebGL, it doesn't really matter which.

The chromium implementation does re-raster content at the exact resolution of the copy, but that's not enough to prevent fuzziness.

Thank you for the explanation, makes sense!

If I understand you correctly, what you're asking for is a way to get the intrinsic destination size that would be computed by a call to copyElementImageToTexture(), for the purpose of sizing the destination texture. I don't have a strong feeling about that, but I would point out that it's pretty sugary syntax for something that can be trivially poly-filled.

That's what I'm asking yes. But the simple polyfill isn't exact, right? To get the exact dimensions you'd need ResizeObserver (which is async). OTOH, maybe it's inherently asynchronous so that's the best we can do, I'm obviously not that familiar with layout.

Even ResizeObserver is not authoritative here, because it will not account for any source rect parameters you might specify, and also because the pixel-snapping logic for web content is not identical to the intrinsic size calculation used for copyElementImageToTexture().

The polyfill I gave above exactly mirrors the intrinsic size calculation, but I suppose there is still a risk of inaccuracy due to numerical precision. For example, the internal calculation might get a single-precision float value of 99.00000000, while the double-precision arithmetic in javascript might give 99.00000000001 due to the extra precision, and when you ceil() these two values you will get 99 and 100 respectively. I'm not sure how plausible this scenario is, since, CSS pixel measurements will always be snapped to the sub-pixel layout grid, i.e. 1/64th of a pixel, but I can't rule it out.

Having said that: this seems exceedingly hair-splitting, and I think it obscures the more important point which is that developers should be deliberate in picking their poison between guttering and clipping. To put it another way, it's not the 0.0000000000001 case we should be worrying about -- if the intrinsic width is 99.000000001 and the texture width is 99 due to numerical imprecision, then the result is that we will clip .000000001 of a pixel on the right, which is not something anyone should care about. The more interesting case is 99.4 or 99.6, and in that case we want the developer to make an informed decision about whether to gutter or clip.

On naming: the one constant between the 2D/WebGL/WebGPU names is ElementImage, and I would like to preserve that. I don't really have a preference between drawElementImageToTexture and copyElementImageToTexture. We chose copy to emulate copyExternalImageToTexture, because there is a lot of conceptual commonality between the two, but draw is maybe useful to highlight the contrast.

drawElementImageToTexture SGTM.

I didn't realize ResizeObserver wouldn't work either. Makes sense though.

I agree about the float precision not being a concern, as long as we don't have validation errors that are sensitive to it.

For gutter vs clip, that's basically up to whether the developer calls ceil or floor right? But we can't really enforce that they call either one, whatever is passed will get rounded to an integer. Unless we take the size as a double and the API does the ceil so they gutter by default, and have to floor if they want to clip. Which, actually, now that I'm saying, seems to make sense?

• The source size parameter specifies the clip rectangle in CSS space
• The destination size parameter specifies the size to rasterize it to, this doesn't fundamentally have to be an integer size
• We infer the destination pixel rectangle size from that using ceil

Of course this is probably something you would want to do in all the APIs so it's not a trivial change. But just an idea.

In the parent proposal there's this thing about the element being a descendant of the canvas it's being drawn to. That doesn't make sense in WebGPU because WebGPU can have any number of canvases (that is, 0 or more).

We need to flesh out what the constraint is - is it, you can use an element with a device if it's a child of any layoutsubtree canvas that is currently attached to this device? Or would we just not have this constraint for WebGPU (it can be a child of any layoutsubtree canvas with context type "webgpu"? or any layoutsubtree canvas at all?)

From an implementation standpoint, I don't think there's any need for the source to be associated with a canvas attached to this device. I think this is more like copyExternalImageToTexture, where there are no real constraints on the source image.

My understanding is that "you can use an element with a device if it's a child of any layoutsubtree canvas at all". The child-of-canvas requirement is to ensure that various properties of the rendered content are guaranteed, particularly related to privacy controls and CSS transforms and positioning. It doesn't seem like it would matter which canvas.