Pass --enable-experimental-web-platform-features to Chrome Dev

[foolip]

In Intent to Ship: Double-position gradient color stop syntax we noticed that Chrome Dev on wpt.fyi isn't running with experimental web platform features enabled, based on these results:
https://wpt.fyi/results/css/css-images/gradient/color-stops-parsing.html?label=experimental

This is a request to add that.

Note that for regression detection, there may at a future point also be necessary to run Chrome Dev (or Beta) without the flag, to match stable, but we can deal with that later. @lukebjerring FYI.

[jugglinmike]

In gh-552, @jgraham wrote:

In general I wish we would put these flags in wptrunner directly rather than having special configuration here.

Writing in gh, James continued:

So it seems like the high impact items here might just be

• Version detect the Chrome binary and run dev/canary versions with the --enable-experimental-web-platform-features flag
• Produce a minimal version of the Firefox prefs file with no feature additions (in Mozilla central) and use that for stable Firefox releases.

@jgraham I'm not sure if you were recommending that these things be done unconditionally or only in the presence of a new --experimental flag. The former seems preferable because it doesn't preclude @foolip's (admittedly uncertain) use case and because it avoids surprising folks who expect the default configuration when running without flags. Could you clarify?

[lukebjerring]

FWIW I think that we're kinda blocked on web-platform-tests/wpt.fyi#212 - running with and without experimental flags can be added as metadata using labels on the runs.

[jugglinmike]

We may not be blocked. That feature is about annotating results. While a results collector might need a way to communicate that the experimental flag was enabled, it also needs to actually enable that flag. We can enable the flag for the results we're collecting today and include the new "experimental" label whenever the results receiver supports it.

[lukebjerring]

Right - I thought blocked because the issue is around the [browser]-experimental TestRun entries 'clashing' with each other (i.e. with and without the experimental flag not being distinct in wpt.fyi's current infrastructure). Maybe an extra suffix in the name could fix that.

[Hexcles]

Now that the receiver can receive labels from runners, I think it's a good time to flip the switch.

In web-platform-tests/wpt.fyi#252 (comment) , I proposed to use the "experimental" label to mean "the bleeding-edge version + experimental features" combo.

IIUC Firefox prefs already enable experimental features, so the only remaining task is to pass the --enable-experimental-web-platform-features flag to Chrome Dev (not sure whether that'd be done in wptrunner or here though). The runner already starts sending the "experimental" label to the staging receiver for experimental-channel browsers (#575).

If later we find needs for running other configurations (e.g. experimental features on stable or dev without experimental features), we can come up with other labels for that. The beauty of a label system (as opposed to a category or suffix system) is the liberty to mix and match. I'd like to use the nice and short name "experimental" for the most common use case :)

[foolip]

Ping @jugglinmike, what's the status of this? I don't think this should be blocked on web-platform-tests/wpt#10452, FWIW.

[Hexcles]

I was going to send a PR but found @jugglinmike had already done it in 47a63f9

So looks like we can close the issue.

(I'm not sure if the change has been deployed, but either way we'll soon have a deployment & restart because of #576 .)

[jugglinmike]

I've been a bit secretive because I haven't found a good opportunity to deploy that change. I wanted to wait until we'd successful collected results with the flag enabled before considering this resolved.

Sorry for the confusion!

[foolip]

This will have been fixed by web-platform-tests/wpt#13011.