tld service for webextensions

👁 Image	Shane Caraveo (:mixedpuppy) Reporter
	Description • 9 years ago

We should consider providing tld for webextensions. The SDK provides it here: https://dxr.mozilla.org/mozilla-central/source/addon-sdk/source/lib/sdk/url.js#313 A simple Host api might be useful, such as https://github.com/NiklasGollenstede/get-tld CookieMonster is one use case.

👁 Image	Marat Tanalin
	Comment 1 • 9 years ago

The `eTLD` service is also used in the UsableHomeButton [1] extension. Namely, the `Services.eTLD.getPublicSuffixFromHost()` function is used. A pure-JS library is not a reasonable equivalent of built-in `eTLD` features: * The huge advantage of the built-in `eTLD` service is that the actual list of top-level domains is automatically always up-to-date thanks to browser's automatic updates. This cannot be achieved in a reasonable way with a library. * Using multiple different implementations of the same feature in the same application leads to inconsistencies and is considered bad practice. * Using an extra library while already having similar functionality in the application results in wasting memory and CPU power and generally have a negative performance impact. [1] https://addons.mozilla.org/firefox/addon/usablehomebutton/

👁 Image	:shell escalante
	Updated • 9 years ago

👁 Image	Andy McKay
	Updated • 9 years ago

👁 Image	Caitlin Neiman [:caitmuenster]
	Comment 2 • 9 years ago

To be discussed at Nov. 29, 2016 WE triage meeting. Agenda: https://docs.google.com/document/d/1IMBFXHNpg_A-15VdJM1Hh8DUUXF1xNFy87W1w8ZEOBk/edit?usp=sharing

👁 Image	Shane Caraveo (:mixedpuppy) Reporter
	Comment 3 • 9 years ago

We've had more than one request for this, I think providing an API to the internal implementation is better than making people use external libraries.

👁 Image	Kris Maglione [:kmag]
	Comment 4 • 9 years ago

We discussed this and decided that it should probably be implemented as an extension-specific WebIDL interface, probably along the lines of MozURLUtils.

👁 Image	Martin Kimmerle
	Comment 5 • 9 years ago

Good to see this feature will be implemented. Is there a schedule? The RequestPolicy add-on extensively uses the public suffix list. According to the WE triage meeting, the priority should be P2, no? Thanks.

👁 Image	Andy McKay
	Comment 6 • 9 years ago

A good catch, the meeting did say P2, changing. There's no schedule for this yet, its in the backlog. If its something you'd like to contribute code to, please let us know.

👁 Image	Hector Zhao [:hectorz]
	Updated • 9 years ago

👁 Image	:shell escalante
	Updated • 9 years ago

👁 Image	Tomislav Jovanovic :zombie
	Updated • 9 years ago

👁 Image	Tomislav Jovanovic :zombie
	Comment 7 • 9 years ago

(In reply to Kris Maglione [:kmag] from comment #4) > We discussed this and decided that it should probably be implemented as an > extension-specific WebIDL interface, probably along the lines of MozURLUtils. To confirm I understand the intent here, the idea is to have a `MozURLUtils` property on the `window` global, available only to web extensions (for now), which would include (for this bug), a method like `getTLD(url)`?

👁 Image	Kris Maglione [:kmag]
	Comment 8 • 9 years ago

Window and Sandbox globals, yes. Or something along those lines, anyway.

👁 Image	Franz Erlendorf
	Comment 9 • 9 years ago

Getting the list of TLDs can be done by the extension itself, an API provided by the browser is not needed. Extensions can cache the list, thereby having offline access to the list.

👁 Image	tabmix.onemen
	Updated • 8 years ago

👁 Image	Anthony Lieuallen
	Comment 11 • 8 years ago

+1 for this. Greasemonkey uses nsIEffectiveTLDService in our legacy version.

👁 Image	Mike Hommey [:glandium] OoO until Jun 18
	Comment 12 • 8 years ago

This would be useful for features I want to add to the tab-stat addon (grouping by domain name). Each addon reimplementing something the browser already has built-in (with automatic updates of the list, at that), would be error prone and suboptimal.

👁 Image	Marat Tanalin
	Comment 13 • 8 years ago

I’m forced to temporarily remove the eTLD-powered feature entirely from WebExtensions version of my UsableHomeButton extension since bundling an own copy of 12000+-entries database with each extension that needs it looks unreasonable at least in terms of performance.

👁 Image	sgl4kn
	Comment 14 • 8 years ago

This would be great, millions of users use currently sth. like this extensive Adblock implementation (with other addons using different and even slower approaches): var publicSuffixes = { "0.bg": 1, "1.bg": 1, "1kapp.com": 1 }; function getDomain(hostname) { let bits = hostname.split("."); let cutoff = bits.length - 2; for (let i = 0, bitsLen = bits.length; i < bitsLen; i++){ let offset = publicSuffixes[bits.slice(i).join(".")]; if (typeof offset != "undefined"){ cutoff = i - offset; break; } } if (cutoff <= 0) return hostname; return bits.slice(cutoff).join("."); }

👁 Image	David Durst [:ddurst]
	Updated • 8 years ago

👁 Image	Luca Greco [:rpl] [:luca] [:lgreco]
	Updated • 8 years ago

👁 Image	BMO Automation
	Updated • 7 years ago

👁 Image	Shane Caraveo (:mixedpuppy) Reporter
	Comment 15 • 6 years ago

ni? Fallen to consider and fit in somewhere.

👁 Image	Philipp Kewisch [:Fallen][☁️📆]
	Comment 16 • 6 years ago

Thanks for tagging me. I'm fine with having this sort of API available, though I'm not sure about the approach. If we are going to be exposing this on the window I'd prefer if we could work towards standardizing it instead of having a WebExtensions specific window property.

👁 Image	Luca Greco [:rpl] [:luca] [:lgreco]
	Updated • 6 years ago

👁 Image	sgl4kn
	Comment 17 • 5 years ago

BTW: there are also Rust implementations in an attempt to parse PSL quickly:
https://crates.io/crates/psl

There are hundreds of JS libraries and all big extensions like Adblock Plus parse the PSL multiple times on each link on a every website. However, these are all not very efficient.

I use a Regex in my scripts that is executed on every website link. This Regex checks if a link is a referrer link (e.g. http://www.yahoo.de/redirect.cgi?=www.standard.co.uk). For different conditions, the PSL has to be parsed 4 times on each link on a website.
Adbuster's reference implementation is using an array map which results in an overall performance loss of 20% for each website, but it would be awesome if there would be a faster, FF integrated solution.

👁 Image	Danny Colin [:sdk]
	Updated • 4 years ago

👁 Image	Danny Colin [:sdk]
	Updated • 4 years ago

👁 Image	Francis :mckenfra Assignee
	Comment 18 • 4 years ago
obsolete

Comment hidden (obsolete)

Exposes a call to Services.eTLD.getBaseDomain(url) as a new
webextensions API method: browser.dns.getPublicSuffix(url)

E.g. calling this method with "https://www.mozilla.org" results in "mozilla.org"

This will save Addon creators from having to reinvent the wheel.
The Mozilla Multi-Account Containers Addon team intends to use this functionality
for a Wildcard Subdomains feature that is currently in development, see PR:
https://github.com/mozilla/multi-account-containers/pull/2352

👁 Image	Phabricator Automation
	Updated • 4 years ago

👁 Image	Francis :mckenfra Assignee
	Comment 19 • 4 years ago
obsolete

Comment hidden (obsolete)

Depends on D148988

👁 Image	Rob Wu [:robwu]
	Comment 20 • 4 years ago

I filed an issue to discuss this feature in the WebExtensions Community group:

https://github.com/w3c/webextensions/issues/231
("Extension API to find the public suffix (eTLD) of a given URL/domain")

👁 Image	Phabricator Automation
	Updated • 4 years ago

👁 Image	Francis :mckenfra Assignee
	Comment 21 • 3 years ago

Exposes a call to Services.eTLD.getPublicSuffix(url) as a new
webextensions API method: browser.dns.getPublicSuffix(url)

E.g. getPublicSuffix("https://www.mozilla.co.uk:80") => "co.uk"

The method also takes an optional "additionalParts" integer parameter:

E.g. getPublicSuffix("https://www.mozilla.co.uk:80", 1) => "mozilla.co.uk"

This will save Addon creators from having to reinvent the wheel.
The Mozilla Multi-Account Containers Addon team intends to use this functionality
for a Wildcard Subdomains feature that is currently in development, see PR:
https://github.com/mozilla/multi-account-containers/pull/2352

👁 Image	Phabricator Automation
	Updated • 3 years ago

👁 Image	BugBot [:suhaib / :marco/ :calixte]
	Comment 22 • 3 years ago
off-topic

Comment hidden (off-topic)

The following patch is waiting for review from inactive reviewers:

ID	Title	Author	Reviewer Status
D149692	Bug 1315558 - getPublicSuffix(url) for webextensions. r=rpl,robwu	3ecdbelo	rpl: Inactive
robwu: Inactive

:3ecdbelo, could you please find another reviewer?

For more information, please visit auto_nag documentation.

👁 Image	Suhaib Mujahid [:suhaib]
	Comment 23 • 3 years ago
off-topic

Comment hidden (off-topic)

Sorry for the noise, there was a problem with a Bugzilla API that caused the bot to think more people were inactive than normal (bug 1789646).

👁 Image	BMO Automation
	Updated • 3 years ago

👁 Image	BugBot [:suhaib / :marco/ :calixte]
	Comment 24 • 3 years ago

The severity field for this bug is relatively low, S3. However, the bug has 277 votes and 5 See Also bugs.
:3ecdbelo, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.

👁 Image	BugBot (nomail) [:suhaib / :marco/ :calixte]
	Comment 25 • 3 years ago

The last needinfo from me was triggered in error by recent activity on the bug. I'm clearing the needinfo since this is a very old bug and I don't know if it's still relevant.

👁 Image	BugBot [:suhaib / :marco/ :calixte]
	Comment 26 • 3 years ago

The bug assignee is inactive on Bugzilla, so the assignee is being reset.

👁 Image	Sylvestre Ledru [:Sylvestre]
	Updated • 2 years ago

👁 Image	Rob Wu [:robwu]
	Comment 27 • 1 year ago • Edited

:mcfrenka

After some discussions in the WECG there is a generally positive sentiment towards a (cross-browser) extension API for public suffix. Are you interested in creating a proposal in the WECG so that we can finalize the details of the API? Once that is done the patch can be tweaked based on the feedback and landed.

The proposal process is described at https://github.com/w3c/webextensions/blob/main/proposals/proposal_process.md

I mentioned this status at https://github.com/w3c/webextensions/issues/231#issuecomment-2247438680

👁 Image	Phabricator Automation
	Updated • 1 year ago

👁 Image	Francis :mckenfra Assignee
	Comment 28 • 1 year ago

:robwu

Thanks for pushing this forward. Yes I would be interested in creating a WECG proposal. What sort of timeframe would be good (for creating the proposal)?

👁 Image	Rob Wu [:robwu]
	Comment 29 • 1 year ago

There is no time pressure here. It is totally fine if you need a few weeks to find time. Feel free to contact me on Matrix if you have some quick questions.

👁 Image	BugBot (nomail) [:suhaib / :marco/ :calixte]
	Comment 30 • 1 year ago

The bug assignee is inactive on Bugzilla, so the assignee is being reset.

👁 Image	BugBot [:suhaib / :marco/ :calixte]
	Comment 31 • 1 year ago

Clear a needinfo that is pending on an inactive user.

Inactive users most likely will not respond; if the missing information is essential and cannot be collected another way, the bug maybe should be closed as INCOMPLETE.

For more information, please visit BugBot documentation.

👁 Image	Francis :mckenfra Assignee
	Comment 32 • 1 month ago

👁 Image	Phabricator Automation
	Updated • 1 month ago

👁 Image	Francis :mckenfra Assignee
	Comment 33 • 1 month ago

Depends on D295511

👁 Image	Francis :mckenfra Assignee
	Comment 34 • 1 month ago

Depends on D295512

👁 Image	Francis :mckenfra Assignee
	Comment 35 • 1 month ago

Depends on D295513

👁 Image	Phabricator Automation
	Updated • 1 month ago

👁 Image	Phabricator Automation
	Updated • 22 days ago

👁 Image	Phabricator Automation
	Updated • 22 days ago

👁 Image	Francis :mckenfra Assignee
	Comment 36 • 17 days ago

Depends on D295513

👁 Image	Phabricator Automation
	Updated • 16 days ago

👁 Image	Phabricator Automation
	Updated • 16 days ago

👁 Image	Pulsebot
	Comment 37 • 13 days ago

Pushed by rob@robwu.nl: https://github.com/mozilla-firefox/firefox/commit/ad0bfcf7bc08 https://hg.mozilla.org/integration/autoland/rev/d7930fce5f5f Add publicSuffix extension API. 1/4 - scaffolding. r=robwu,webidl,smaug https://github.com/mozilla-firefox/firefox/commit/800292d8c79b https://hg.mozilla.org/integration/autoland/rev/cbaac863416c Add publicSuffix extension API. 2/4 - core implementation. r=robwu https://github.com/mozilla-firefox/firefox/commit/81c6595869fb https://hg.mozilla.org/integration/autoland/rev/3acb0ec288ca Add publicSuffix extension API. 3/4 - tests. r=robwu https://github.com/mozilla-firefox/firefox/commit/256e60ec9434 https://hg.mozilla.org/integration/autoland/rev/19d514f906a1 Add publicSuffix extension API. 4/4 - rename encoding=unicode. r=robwu

👁 Image	Treeherder Bug Filer
	Updated • 13 days ago

👁 Image	Cosmin Sabou [:CosminS]
	Comment 38 • 13 days ago
bugherder

https://hg.mozilla.org/mozilla-central/rev/d7930fce5f5f
https://hg.mozilla.org/mozilla-central/rev/cbaac863416c
https://hg.mozilla.org/mozilla-central/rev/3acb0ec288ca
https://hg.mozilla.org/mozilla-central/rev/19d514f906a1

👁 Image	Rob Wu [:robwu]
	Updated • 12 days ago

👁 Image	Rob Wu [:robwu]
	Comment 39 • 12 days ago

Thanks a lot Francis for your contribution here. This marks the completion of one of the most voted on bugs in WebExtensions on Bugzilla!

:mckenfra contributed the following significant pieces:

• Public Suffix API proposal document: https://github.com/w3c/webextensions/blob/main/proposals/public-suffix.md
• Public Suffix API in Firefox (patches above).

Through the effort he showed a lot of patience, since it took over a year between the start of the proposal and the final approvals. But the cross-browser alignment and community efforts paid off, because we now have a cross-browser publicSuffix extension API implementation, including a late fixup to the API (part 4 patch of the patch stack, renaming "unicode" to "display") following cross-browser implementer experience that highlighted an issue (https://github.com/w3c/webextensions/pull/1013).

👁 Image	Phabricator Automation
	Updated • 12 days ago

👁 Image	rbloor
	Comment 40 • 8 days ago

Documentation updates in:

• (BCD) Bug 1315558 webExtensions Public Suffix API #29822
• (content) Bug 1315558 documentation for the publicSuffix API #44401

👁 Image	rbloor
	Updated • 12 hours ago