Grant user gesture activation when interacting with the webpage

[beaufortfrancois]

Even though the WebMCP proposal has a requestUserInteraction() agent method to asynchronously seek user input during the execution of a tool, I wonder if we should also grant user gesture activation when interacting with the webpage from a WebMCP tool execute() function.

https://developer.mozilla.org/en-US/docs/Web/Security/Defenses/User_activation#transient_activation is a non-exhaustive list of Web APIs gated by transient activation, which means those don't work for now in WebMCP.

One thing worth noting is that some Web APIs actually consume user activation.

[johannhof]

Does an agent "interact" with the website through the execute function? Doesn't that just trigger JS to be run by the website itself? I might be misinformed here. What is a use case that you had in mind?

[beaufortfrancois]

I had this prototype idea where I'd control a video player from WebMCP tool and requestPictureInPicture() didn't work because it requires transient activation.

[johannhof]

Hmm I'm not sure just blanket granting scripts run by agents transient activation is the right solution for that, and it might open some abuse potential. I think Web Permissions, Picture in Picture and things like popup blocking should have custom solutions that ensure that the agent can properly integrate with whatever happens. Most often activation is required because something will happen in the UI, and most agents won't have (and probably should not have) the ability to control it, e.g. accept permission prompts. So they would probably benefit from a more bespoke / custom solution.

For requestPictureInPicture it might be fine to say if it was called from a WebMCP execution the user gesture requirement is waived.

[khushalsagar]

Most often activation is required because something will happen in the UI, and most agents won't have (and probably should not have) the ability to control it, e.g. accept permission prompts.

Sounds like you're saying there will be multiple cases where granting transient activation won't resolve the issue since the user will have to consent to a browser-provided permission dialog anyway? In that case the author has to ask the user to interact and unblock the action anyway. I'm assuming in that case the browser can bring user attention to interact with the necessary prompt. And in some cases this permission state is persisted by the browser, like always allowing an origin to use picture-in-picture.

We could go through the cases one-by-one but it's an interesting fundamental question. If an agent starts interacting with the site it was likely triggered by the user, should it get user activation? I think yes. I'm not sure about renewing the user activation without user interaction though.

[johannhof]

As you note, you're definitely going to have open questions around potential abuse, whether user activation should be consumed, etc.

I'm also not sure that simply asking the user to accept the regular browser permission prompt is an ideal user experience for most browsing agents, especially when the requested permission is unnecessary for the task.

I'm not fundamentally against this, and it would also solve e.g. opening popups, just want to call out that it leaves a lot of open questions in my mind :)