If there are any googlers here, I'd like to report an even more dangerous website. As much as 30-50% of the traffic to it relates to malware or scams, and it has gone unpunished for a very long time.
What i really don't understand at least here in Europe the advertising partner (adsense) must investigate at least minimally whether the advertising is illegal or fraudulent, i understand that sites.google etc are under "safe harbor" but that's not the point with adsense since people from google "click" the publish button and also get money to publish that ad.
The law is only for plebs like you and me. Companies get a pass.
I'm still amazed how deploying spyware would've rightfully landed you in jail a couple decades back, but do the same thing on the web under the justification of advertising/marketing and suddenly it's ok.
I see the same scam/deepfake ad(s) pretty much persistently. Maybe they actually differ slightly (they are AI gen mostly), but it's pretty obvious what they are, and I'm sure they get flagged a lot.
They just need to introduce a basic deposit to post ads, and you lose it if you put up a scam ad. Would soon pay for the staff needed to police it, and prevent scammers from bypassing admin by trivially creating new accounts.
You know what? I don't even mind them killing it, because of course there are a whole pile of items under the anti-trust label that google is doing so why not one more. But what I do take issue with is the gaslighting, their attempt to make the users believe that this is in the users interests, rather than in google's interests.
If we had functional anti-trust laws then this company would have been broken up long ago, Alphabet or not. But they keep doing these things because we - collectively - let them.
I know they won't. But we have all the tools to force them to care. We just don't use the tools effectively, and between that and lobbying they get a free pass to pretty much do as they please.
Use one of the forks. librewolf, waterfox, zen. Firefox itself lost trust when Mozilla tried to push the new Terms of Use earlier this year. That was so aggressively user-hostile that nobody should trust Mozilla ever again. Using a fork puts an insulation layer between you and Mozilla.
Librewolf is just a directly de-mozillaed and privacy-enhanced Firefox, similar to Ungoogled Chromium. I've been trying to get in the habit of using Zen Browser, which has a bunch of UI changes.
The problem is that all those forks are beholden to Mozilla's corporate interests the same way the chromium derivatives are beholden to Google's corporate interests. What we need is one of the newer independent engines to mature - libweb, servo or blitz.
Apparently the "best practise" is using Manifest V3 versus V2.
Reading a bit online (not having any personal/deep knowledge) it seems the original extension also downloaded updates from a private (the developers) server, while that is no longer allowed - they now need to update via the chrome extension, which also means waiting for code review/approval from google.
I can see the security angle there, it is just awkward how much of an vested interest google has in the whole topic. ad-blocking is already a grey area (legally), and there is a cat-and-mouse between blockers and advertisers; it's hard to believe there is only security best-practise going on here.
The same outfit is runimg a domain called blogger.
Reminds me of MS blocking a website of mine for dangerous script. The offending thing i did was use document.write to put copyright 2025 (with the current year) at the end of static pages.
Microsoft's own Outlook.com flags Windows Insider emails coming from a .microsoft.com domain as junk even after marking the domain as "no junk". They know themselves well.
The integrated button to join a Microsoft Teams meeting directly from my Microsoft Outlook Calendar doesn't work because Microsoft needs to scan the link from Microsoft to Microsoft for malware before proceeding, and the malware scanning service has temporary downtime and serves me static page saying "The content you are accessing cannot currently be verified".
sites.google.com is widely abused but so practically any site which allows users to host content of their choice and make it publicly available. Where google can be different is that they famously refuse yo do work which they cannot automate and probably they cannot (or don’t want) to automate detection/blocking of spam/phishing hosted on sites.google.com and processing of abuse reports.
Happened to me last week. One morning we wake up and the whole company website does not work.
Not advice with some time to fix any possible problem, just blocked.
We gave very bad image to our clients and users, and had to give explanations of a false positive from google detection.
The culprit, according to google search console, was a double redirect on our web email domain (/ -> inbox -> login).
After just moving the webmail to another domain, removing one of the redirections just in case, and asking politely 4 times to be unblocked.. took about 12 hours. And no real recourse, feedback or anything about when its gonna be solved. And no responsibility.
The worse is the feeling of not in control of your own business, and depending on a third party which is not related at all with us, which made a huge mistake, to let out clients use our platform.
I'm beginning to seriously think we need a new internet, another protocol, other browsers just to break up the insane monopolies that has been formed, because the way things are going soon all discourse will be censored, and competitors will be blocked soon.
We need something that's good for small and medium businesses again, local news and get an actual marketplace going - you know what the internet actually promised.
It's very, very hard to overcome the gravitational forces which encourage centralization, and doing so requires rooting the different communities that you want to exist in their own different communities of people. It's a political governance problem, not a technical one.
The community around NOSTR are basically building a kind of semantic web, where users identities are verified via their public key, data is routed through content agnostic relays, and trustworthiness is verified by peer recommendation.
They are currently experimenting with replicating many types of services which are currently websites as protocols with data types, with the goal being that all of these services can share available data with eachother openly.
It's definitely more of a "bazaar" model over a "catherdral" model, with many open questions and it's also tough to get a good overview of what is really going on there. But at least it's an attempt.
It will take the same or less amount of time, to get where we are with current Web.
What we have is the best sim env to see how stuff shape up. So fixing it should be the aim, avoiding will get us on similar spirals. We'll just go on circles.
It won't get anywhere unless it addresses the issue of spam, scammers, phishing etc. The whole purpose of Google Safe Browsing is to make life harder for scammers.
I'm afraid this can't be built on the current net topology which is owned by the Stupid Money Govporation and inherently allows for roadblocks in the flow of information. Only a mesh could solve that.
But the Stupid Money Govporation must be dethroned first, and I honestly don't see how that could happen without the help of an ELE like a good asteroid impact.
If you're going to host user content on subdomains, then you should probably have your site on the Public Suffix List https://publicsuffix.org/list/ .
That should eventually make its way into various services so they know that a tainted subdomain doesn't taint the entire site....
In the past, browsers used an algorithm which only denied setting wide-ranging cookies for top-level domains with no dots (e.g. com or org). However, this did not work for top-level domains where only third-level registrations are allowed (e.g. co.uk). In these cases, websites could set a cookie for .co.uk which would be passed onto every website registered under co.uk.
Since there was and remains no algorithmic method of finding the highest level at which a domain may be registered for a particular top-level domain (the policies differ with each registry), the only method is to create a list. This is the aim of the Public Suffix List.
(https://publicsuffix.org/learn/)
So, once they realized web browsers are all inherently flawed, their solution was to maintain a static list of websites.
God I hate the web. The engineering equivalent of a car made of duct tape.
> Since there was and remains no algorithmic method of finding the highest level at which a domain may be registered for a particular top-level domain
A centralized list like this not just for domains as a whole (e.g. co.uk) but also specific sites (e.g. s3-object-lambda.eu-west-1.amazonaws.com) is both kind of crazy in that the list will bloat a lot over the years, as well as a security risk for any platform that needs this functionality but would prefer not to leak any details publicly.
We already have the concept of a .well-known directory that you can use, when talking to a specific site. Similarly, we know how you can nest subdomains, like c.b.a.x, and it's more or less certain that you can't create a subdomain b without the involvement of a, so it should be possible to walk the chain.
Example:
c --> https://b.a.x/.well-known/public-suffix
b --> https://a.x/.well-known/public-suffix
a --> https://x/.well-known/public-suffix
Maybe ship the domains with the browsers and such and leave generic sites like AWS or whatever to describe things themselves. Hell, maybe that could also have been a TXT record in DNS as well.
> any platform that needs this functionality but would prefer not to leak any details publicly.
I’m not sure how you’d have this - it’s for the public facing side of user hosted content, surely that must be public?
> We already have the concept of a .well-known directory that you can use, when talking to a specific site.
But the point is to help identify dangerous sites, by definition you can’t just let the sites mark themselves as trustworthy and rotate around subdomains. If you have an approach that doesn’t have to trust the site, you also don’t need any definition at the top level you could just infer it.
Not really. Does the car still drive? That sounds like a software bug; hardly indicative that the entire car is held together with duct tape, but a pretty bad bug non the less.
A part of the issue is IMO that browsers have become ridiculously bloated everything-programs. You could take about 90% of that out and into dedicated tools and end up with something vastly saner and safer and not a lot less capable for all practical purposes. Instead, we collectively are OK with frosting this atrocious layer cake that is today's web with multiple flavors of security measures of sometimes questionable utility.
"You could take about 90% of that out and into dedicated tools "
But then you would loose plattform independency, the main selling point of this atrocity.
Having all those APIs in a sandbox that mostly just work on billion devices is pretty powerful and a potential succesor to HTML would have to beat that, to be adopted.
The best thing to happen, that I can see, is that a sane subset crystalizes, that people start to use dominantly, with the rest becoming legacy, only maintained to have it still working.
But I do dream of a fresh rewrite of the web since university (and the web was way slimmer back then), but I got a bit more pragmatic and I think I understood now the massive problem of solving trusted human communication better. It ain't easy in the real world.
But do we need e.g serial port or raw USB access straight from a random website? Even WebRTC is a bit of a stretch. There is a lot of cruft in modern browsers that does little except increase attack surface.
This all just drives a need to come up with ever more tacked-on protection schemes because browsers have big targets painted on them.
WebRTC I use since many years and would miss it a lot. P2P is awesome.
WebUSB I don't use or would miss it right now, but .. the main potential use case is security and it sounds somewhat reasonable
"Use in multi-factor authentication
WebUSB in combination with special purpose devices and public identification registries can be used as key piece in an infrastructure scale solution to digital identity on the internet."
You remove that, and videoconferencing (for business or person to person) has to rely on downloading an app, meaning whoever is behind the website has to release for 10-15 OSes now. Some already do, but not everyone has that budget so now there's a massive moat around it.
> But do we need e.g serial port or raw USB access straight from a random website
Being able to flash an IoT (e.g. ESP32) device from the browser is useful for a lot of people. For the "normies", there was also Stadia allowing you to flash their controller to be a generic Bluetooth/usb one on a website, using that webUSB. Without it Google would have had to release an app for multiple OSes, or more likely, would have just left the devices as paperweights. Also, you can use FIDO/U2F keys directly now, which is pretty good.
Browsers are the modern Excel, people complain that they do too much and you only need 20%. But it's a different 20% for everyone.
Every decent host OS already has a dedicated driver stack to provide game controller input to applications in a useful manner. Why the heck would you ship a reimplementation of that in JS in a website?
> Having all those APIs in a sandbox that mostly just work on billion devices is pretty powerful and a potential succesor to HTML would have to beat that, to be adopted.
I think the giant major downside, is that they've written a rootkit that runs on everything, and to try to make up for that they want to make it so only sites they allow can run.
It's not really very powerful at all if nobody can use it, at that point you are better off just not bothering with it at all.
The Internet may remain, but the Web may really be dead.
> to try to make up for that they want to make it so only sites they allow can run
What do you mean, you can run whatever you want on localhost, and it's quite easy to host whatever you want for whoever you want too. Maybe the biggest modern added barrier to entry is that having TLS is strongly encouraged/even needed for some things, but this is an easily solved problem.
>A part of the issue is IMO that browsers have become ridiculously bloated everything-programs.
I don't see how that solves the issue that PSL tries to fix. I was a script kiddy hosting neopets phishing pages on free cpanel servers from <random>.ripway.com back in 2007. Browsers were way less capable then.
PSL and the way cookies work is just part of the mess. A new approach could solve that in a different way, taking into account all the experience we had with scriptkiddies and professional scammers and pishers since then. But I also don't really have an idea where and how to start.
And of course, if the new solution completely invalidates old sites, it just won't get picked up. People prefer slightly broken but accessible to better designed but inaccessible.
> People prefer slightly broken but accessible to better designed but inaccessible.
We live in world where whatever faang adopts is de facto a standard. Accessible these days means google/gmail/facebook/instagram/tiktok works. Everything else is usually forced to follow along.
People will adopt whatever gives them access to their daily dose of doomscrolling and then complain about rather crucial part of their lives like online banking not working.
> And of course, if the new solution completely invalidates old sites, it just won't get picked up.
Old sites don't matter, only high-traffic sites riddled with dark patterns matter. That's the reality, even if it is harsh.
> People prefer slightly broken but accessible to better designed but inaccessible.
It's not even broken as the edge cases are addressed by ad-hoc solutions.
OP is complaining about global infrastructure not having a pristine design. At best it's a complain over a desirable trait. It's hardly a reason to pull the Jr developer card and mindlessly advocate for throwing everything out and starting over.
Are you saying we should make a <Unix Equivalent Of A Browser?> A large set of really simple tools that each do one thing really really really pedantically well?
This might be what's needed to break out of the current local optimum.
I think it's somewhat tribal webdev knowledge that if you host user generated content you need to be on the PSL otherwise you'll eventually end up where Immich is now.
I'm not sure how people not already having hit this very issue before is supposed to know about it beforehand though, one of those things that you don't really come across until you're hit by it.
Don't get me wrong, Google is bad/evil in many ways, but the public suffix list exists to solve a real risk to users. Google is flagging this for a legit reason in this particular case.
> people keep using Google's browser because the safe search guards catch more bad actors than they false positive good actors.
This is the first thing i disable in Chrome, Firefox and Edge. The only safe thing they do is safely sending all my browsing history to Google or Microsoft.
Looking through some of the links in this post, I there are actually two separate issues here:
1. Immich hosts user content on their domain. And should thus be on the public suffic list.
2. When users host an open source self hosted project like immich, jellyfin, etc. on their own domain it gets flagged as phishing because it looks an awful lot like the publicly hosted version, but it's on a different domain, and possibly a domain that might look suspicious to someone unfamiliar with the project, because it includes the name of the software in the domain. Something like immich.example.com.
The first one is fairly straightforward to deal with, if you know about the public suffix list. I don't know of a good solution for the second though.
I don't think the Internet should be run by being on special lists (other than like, a globally run registry of domain names)...
I get that SPAM, etc., are an issue, but, like f* google-chrome, I want to browse the web, not some carefully curated list of sites some giant tech company has chosen.
A) you shouldn't be using google-chrome at all B) Firefox should definitely not be using that list either C) if you are going to have a "safe sites" list, that should definitely be a non-profit running that, not an automated robot working for a large probably-evil company...
> I don't think the Internet should be run by being on special lists
People are reacting as if this list is some kind of overbearing way of tracking what people do on the web - it's almost the opposite of that. It's worth clarifying this is just a suffix list for user-hosted content. It's neither a list of user-hosted domains nor a list of safe websites generally - it's just suffixes for a very small specific use-case: a company providing subdomains. You can think of this as a registry of domain sub-letters.
For instance:
- GitHub.io is on the list but GitHub.com is not - GitHub.com is still considered safe
- I self-host an immich instance on my own domain name - my immich instance isn't flagged & I don't need to add anything to the list because I fully own the domain.
The specific instance is just for Immich themselves who fully own "immich.cloud" but sublet subdomains under it to users.
> *if you are going to have a "safe sites" list"
This is not a safe sites list! This is not even a sites list at all - suffixes are not sites. This also isn't even a "safe" list - in fact it's really a "dangerous" list for browsers & various tooling to effectively segregate security & privacy contexts.
Google is flagging the Immich domain not because it's missing from the safe list but because it has legitimate dangers & it's missing from the dangerous list that informs web clients of said dangers so they can handle them appropriately.
I've coined the phrase "Postel decentralization" to refer to things where people expect there to be some distributed consensus mechanism but it turned out that the design of the internet was to email Jon Postel (https://en.wikipedia.org/wiki/Jon_Postel) to get your name on a list. e.g. how IANA was originally created.
Firefox and Safari also use the list. At least by default, I think you can turn it off in firefox. And on the whole, I think it is valuable to have _a_ list of known-unsafe sites. And note that Safe Browsing is a blocklist, not an allowlist.
The problem is that at least some of the people maintaining this list seem to be a little trigger happy. And I definitely thing Google probably isn't the best custodian of such a list, as they have obvious conflicts of interest.
Oh god, you reminded me the horrors of hosting my own mailserver and all of the white/blacklist BS you have to worry about being a small operator (it's SUPER easy to end up on the blacklists, and is SUPER hard to get onto whitelists)
> I don't know of a good solution for the second though.
I know the second issue can be a legitimate problem but I feel like the first issue is the primary problem here & the "solution" to the second issue is a remedy that's worse than the disease.
The public suffix list is a great system (despite getting serious backlash here in HN comments, mainly from people who have jumped to wildly exaggerated conclusions about what it is). Beyond that though, flagging domains for phishing for having duplicate content smells like an anti-self-host policy: sure there's phishers making clone sites, but the vast majority of sites flagged are going to be legit unless you employ a more targeted heuristic, but doing so isn't incentivised by Google's (or most company's) business model.
The second is a real problem even with completely unique applications. If they have UI portions that have lookalikes, you will get flagged. At work, I created an application with a sign-in popup. Because it's for internal use only, the form in the popup is very basic, just username and password and a button. Safe Browsing continues to block this application to this day, despite multiple appeals.
> When users host an open source self hosted project like immich, jellyfin, etc. on their own domain...
I was just deploying your_spotify and gave it your-spotify.<my services domain> and there was a warning in the logs that talked about thud, linking the issue:
Even the first one only works if there's no need to have site-wide user authentication on the domain, because you can't have a domain cookie accessible from subdomains anymore otherwise.
I thought this story would be about some malicious PR that convinced their CI to build a page featuring phishing, malware, porn, etc. It looks like Google is simply flagging their legit, self-created Preview builds as being phishing, and banning the entire domain. Getting immich.cloud on the PSL is probably the right thing to do for other reasons, and may decrease the blast radius here.
Remember, this is a free service that Google is offering for even their competitors to use.
And it is incredibly valuable thing. You might not think it is, but internet is filled utterly dangerous, scammy, phisy, malwary websites and everyday Safe Browsing (via Chrome, Firefox and Safari - yes, Safari uses Safe Browsing) keeps users safe.
If immich didnt follow best practice that's Google's fault? You're showing your naivety, and bias here.
Please point me to where GoDaddy or any other hosting site mentions public suffix, or where Apple or Google or Mozilla have a listing hosting best practices that include avoiding false positives by Safe Browsing…
>GoDaddy or any other hosting site mentions public suffix
They don't need to mention it because they handle it on behalf of the client. Them recommending best practices like using separate domains makes as much sense as them recommending what TLS configs to use.
>or where Apple or Google or Mozilla have a listing hosting best practices that include avoiding false positives by Safe Browsing…
Since were those sites the go to place to learn how to host a site? Apple doesn't offer anything related to web hosting besides "a computer that can run nginx". Google might be the place to ask if you were your aunt and "google" means "internet" to her. Mozilla is the most plausible one because they host MDN, but hosting documentation on HTML/CSS/JS doesn't necessarily mean they offer hosting advice, any more than expecting docs.djangoproject.com to contain hosting advice.
If you have a service where anyone can sign up and host content on your subdomain, it really is your responsibility to know. Calling this "unfair" because you didn't know is naive.
If amazon shutdown your AWS account, because those same scammers used those domains to host CP rather than phishing pages, would you accept the excuse of "how was I supposed to know?"
Nothing in this article indicates UGC is the problem. It's that Google thinks there's an "official" central immich and these instances are impersonating it.
What malicious UGC would you even deliver over this domain? An image with scam instructiins? CSAM isn't even in scope for Safe Browsing, just phishing and malware.
>You might not think it is, but internet is filled utterly dangerous, scammy, phisy, malwary websites
Google is happy to take their money and show scammy ads. Google ads are the most common vector for fake software support scams. Most people google something like "microsoft support" and end up there. Has Google ever banned their own ad domains?
Google is the last entity I would trust to be neutral here.
The argument would work better if Google wasn't the #1 distributor of scams and malware in the world with adsense. (Which strangely isn't flagged by safe browsing, maybe a coincidence)
> Is that actually relevant when only images are user content?
Yes. For instance in circumstances exactly as described in the thread you are commenting in now and the article it refers to.
Services like google's bad site warning system may use it to indicate that it shouldn't consider a whole domain harmful if it considers a small number of its subdomains to be so, where otherwise they would. It is no guarantee, of course.
Well, using the public suffix list _also_ isolates cookies and treats the subdomains as different sites, which may or may not be desirable.
For example, if users are supposed to log in on the base account in order to access content on the subdomains, then using the public suffix list would be problematic.
Cross domain identity management is a little extra work, but it's far from a difficult problem. I understand the objection to needing to do it when a shared cookie is so easy, but if you want subdomains to be protected from each other because they do not have shared responsibility for each other then it makes sense in terms of privacy & security that they don't automatically share identity tokens and other client-side data.
Browsers already do various levels of isolation based on domain / subdomains (e.g. cookies). PSL tells them to treat each subdomain as if it were a top level domain because they are operated (leased out to) different individuals / entities. WRT to blocking, it just means that if one subdomain is marked bad, it's less likely to contaminate the rest of the domain since they know it's operated by different people.
I think this only is true if you host independent entities. If you simply construct deep names about yourself with demonstrable chain of authority back, I don't think the PSL wants to know. Otherwise there is no hierarchy the dots are just convenience strings and it's a flat namespace the size of the PSLs length.
This is not about user content, but about their own preview environments! Google decided their preview environments were impersonating... Something? And decided to block the entire domain.
There is no law appointing that organization as a world wide authority on tainted/non tainted sites.
The fact it's used by one or more browsers in that way is a lawsuit waiting to happen.
Because they, the browsers, are pointing a finger to someone else and accusing them of criminal behavior. That is what a normal user understands this warning as.
Turns out they are wrong. And in being wrong they may well have harmed the party they pointed at, in reputation and / or sales.
It's remarkable how short sighted this is, given that the web is so international. Its not a defense to say some third party has a list, and you're not on it so you're dangerous
True, and agreed that lawsuits are likely. Disagree that it's short-sighted. The legal system hasn't caught up with internet technology and global platforms. Until it does, I think browsers are right to implement this despite legal issues they might face.
In what country hasn't the legal system caught up?
The point I raise is that the internet is international. There are N legal systems that are going to deal with this. And in 99% of them this isn't going to end well for Google if plaintiff can show there are damages to a reasonable degree.
It's bonkers in terms of risk management.
If you want to make this a workable system you have to make it very clear this isn't necessarily dangerous at all, or criminal. And that a third party list was used, in part, to flag it. And even then you're impeding visitors to a website with warnings without any evidence that there is in fact something wrong.
If this happens to a political party hosting blogs, it's hunting season.
I love Immich & greatly appreciate the amazing work the team put into maintaining it, but between the OP & this "Cursed Knowledge" page, the apparent team culture of shouting from the rooftops complaints that expose their own ignorance about technology is a little concerning to be honest.
I've now read the entire Cursed Knowledge list & - while I found some of them to be invaluable insights & absolutely love the idea of projects maintaining a public list of this nature to educate - there are quite a few red flags in this particular list.
Before mentioning them: some excellent & valuable, genuinely cursed items: Postgres NOTIFY (albeit adapter-specific), npm scripts, bcrypt string lengths & especially the horrifically cursed Cloudflare fetch: all great knowledge. But...
> Secure contexts are cursed
> GPS sharing on mobile is cursed
These are extremely sane security feature. Do we think keeping users secure is cursed? It honestly seems crazy to me for them to have published these items in the list with a straight face.
> PostgreSQL parameters are cursed
Wherein their definition of "cursed" is that PG doesn't support running SQL queries with more than 65535 separate parameters! It seems to me that any sane engineer would expect the limit to be lower than that. The suggestion that making an SQL query with that many parameters is normal seems problematic.
> JavaScript Date objects are cursed
Javascript is zero-indexed by convention. This one's not a huge red flag but it is pretty funny for a programmer to find this problematic.
> Carriage returns in bash scripts are cursed
Non-default local git settings can break your local git repo. This isn't anything to do with bash & everyone knows git has footguns.
I think the “cursed” part (from the developers point of view) is that some phones do that, some don’t, and if you don’t have both kinds available during testing, you might miss something?
Huh. Maybe? I don't want that information available to apps to spy on me. But I do want full file contents available to some of them.
And wait. Uh oh. Does this mean my Syncthing-Fork app (which itself would never strike me as needing location services) might have my phone's images' location be stripped before making their way to my backup system?
EDIT: To answer my last question: My images transferred via Syncthing-Fork on a GrapheneOS device to another PC running Fedora Atomic have persisted the GPS data as verified by exiftool. Location permissions have not been granted to Syncthing-Fork.
Happy I didn't lose that data. But it would appear that permission to your photo files may expose your GPS locations regardless of the location permission.
With the Nextcloud app I remember having to enable full file permissions to preserve the GPS data of auto-uploaded photos a couple of years ago. Which I only discovered some months after these security changes went into effect on my phone. That was fun. I think Android 10 or 11 introduced it.
Looking now I can't even find that setting anymore on my current phone. But the photos still does have the GPS data intact.
It's not if it silently alters the file.
i do want GPS data for geolocation, so that when i import the images in the right places they are already placed where they should be on the map
Yep, and it's there for very goos reasons. However if you don't know about it, it can be quite surprising and challenging to debug.
Also it's annoying when your phones permissions optimiser runs and removes the location permissions from e.g. Google Photos, and you realise a few months later that your photos no longer have their location.
> JavaScript date objects are 1 indexed for years and days, but 0 indexed for months.
I don't disagree that months should be 1-indexed, but I would not make that assumption solely based on days/years being 1-indexed, since 0-indexing those would be psychotic.
The only reason I can think of to 0-index months is so you can do monthName[date.getMonth()] instead of monthName[date.getMonth() - 1].
I don't think adding counterintuitive behavior to your data to save a "- 1" here and there is a good idea, but I guess this is just legacy from the ancient times.
Why so? Months in written form also start with 1, same as days/years, so it would make sense to match all of them.
For example, the first day of the first month of the first year is 1.1.1 AD (at least for Gregorian calendar), so we could just go with 0-indexed 0.0.0 AD.
As it says, bulk inserts with large datasets can fail. Inserting a few thousand rows into a table with 30 columns will hit the limit. You might run into this if you were synchronising data between systems or running big batch jobs.
Sqlite used to have a limit of 999 query parameters, which was much easier to hit. It's now a roomy 32k.
In the past I've used batches of data, inserted into a separate table with all the constraints turned off and using UNNEST, and then inserted into the final table once it was done. We ended up both batching the data and using UNNEST because it was faster but it still let us resume midway through.
We probably should have been partitioning the data instead of inserting it twice, but I never got around to fixing that.
COPY is likely a better option if you have access to the host, or provider-specific extensions like aws_s3 if you have those. I'm sure a data engineer would be able to suggest a better ETL architecture than "shove everything into postgres", too.
Was MERGE too slow/expensive? We tend to MERGE from staging or temporary tables when we sync big data sets. If we were on postgres I think we'd use ... ON CONFLICT, but MERGE does work.
> PostgreSQL USER is cursed
> The USER keyword in PostgreSQL is cursed because you can select from it like a table, which leads to confusion if you have a table name user as well.
The one thing I never understood about these warnings is how they don't run afoul of libel laws. They are directly calling you a scammer and "attacker". The same for Microsoft with their unknown executables.
They used to be more generic saying "We don't know if its safe" but now they are quite assertive at stating you are indeed an attacker.
"The people living at this address might be pedophiles and sexual predators. Not saying that they are, but if your children are in the vicinity, I strongly suggest you get them back to safety."
Still asserts that in that house there may be sexual predators. If I lived in that house I wouldnt be happy, and I would want a way of clearing the accusations and proving that there are indeed no sexual predators in my house quicksmart before other people start avoiding it.
You can’t possibly use the “they use the word ‘might’” argument and not mention the death red screen those words are printed over. If you are referring to abidance to the law, you are technically right. If we remove the human factor, you technically are.
If the false positive rate is consistently 0.0%, that is a surefire sign that the detector is not effective enough to be useful.
If a false positive is libel, then any useful malware detector would occasionally do libel. Since libel carries enormous financial consequences, nobody would make a useful malware detector.
I am skeptical that changing the wording in the warning resolves the fundamental tension here. Suppose we tone it down: "This executable has traits similar to known malware." "This website might be operated by attackers."
Would companies affected by these labels be satisfied by this verbiage? How do we balance this against users' likelihood of ignoring the warning in the face of real malware?
The problem is that it's so one sided. They do what they want with no effort to avoid collateral damage and there's nothing we can do about it.
They could at least send a warning email to the RFC2142 abuse@ or hostmaster@ address with a warning and some instructions on a process for having the mistake reviewed.
I worked for a company who had this happen to an internal development domain, not exposed to the public internet. (We were doing security research on our own software, so we had a pentest payload hosted on one of those domains as part of a reproduction case for a vulnerability we were developing a fix for.)
Our lawyers spoke to Google's lawyers privately, and our domains got added to a whitelist at Google.
It depends, if it's a clear-cut case, then in jurisdictions with a functioning legal system it can be feasible to sue.
Likewise, if it's a fuckup that just needs to be put in front of someone who cares, a lawsuit is actually a surprisingly effective way of doing that. This moves your problem from "annoying customer support interaction that's best dealt with by stonewalling" into "legal says we HAVE to fix this".
Imagine if you bought a plate at Walmart and any time you put food you bought elsewhere on it, it turned red and started playing a warning about how that food will probably kill you because it wasn't Certified Walmart Fresh™
Now imagine it goes one step further, and when you go to eat the food anyway, your Walmart fork retracts into its handle for your safety, of course.
No brand or food supplier would put up with it.
That's what it's like trying to visit or run non-blessed websites and software coming from Google, Microsoft, etc on your own hardware that you "own".
This is the future. Except you don't buy anything, you rent the permission to use it. People from Walmart can brick your carrots remotely even when you don't use this plate, for your safety ofc
I see a lot of comments here about using some browser that will allow ME to see sites I want to see, but I did not see a lot about how do I protect my site or sites of clients from being subjected to this. Is there anything proactive that can be done? A set of checks almost like regression testing? I understand it can be a bit like virus builders using anti virus to test their next virus. But is there a set of best practices that could give you higher probability of not being blocked?
> how do I protect my site or sites of clients from being subjected to this. Is there anything proactive that can be done?
Some steps to prevent this happening to you:
1. Host only code you own & control on your own domain. Unless...
2. If you have a use-case for allowing arbitrary users to publish & host arbitrary code on a domain you own (or subdomains of), then ensure that domain is a separate dedicated one to the ones you use for your own owned code, that can't be confused with your own owned hosted content.
3. If you're allowing arbitrary members of the public to publish arbitrary code for preview/testing purposes on a domain you own - have the same separation in place for that domain as mentioned above.
4. If you have either of the above two use-cases, publish that separated domain on the Mozilla Public Suffix list https://publicsuffix.org/
That would protect your domains from being poisoned by arbitrary publishing, but wouldn't it risk all your users being affected by one user publishing?
Allowing user publishing is an inherent risk - these are good mitigations but nothing will ever be bulletproof.
The main issue is protecting innocent users from themselves - that's a hard one to generalise solutions to & really depends on your publishing workflows.
Beyond that, the last item (Public Suffix list) comes with some decent additional mitigations as an upside - the main one being that Firefox & Chrome both enable more restrictive cookie settings while browsing any domains listed in the public suffix list.
---
All that said - the question asked in the comment at the top of the thread wasn't about protecting users from security risk, but protecting the domain from being flagged by Google. The above steps should at least do that pretty reliably, barring an actual legitimate hack occurring.
Never host your test environments as Subdomains of your actual production domain.
You'll also run into email reputation as well as cookie hell. You can get a lot of cookies from the production env if not managed well.
This. I cannot believe the rest of the comments on this are seemingly completely missing the problem here & kneejerk-blaming Google for being an evil corp. This is a real issue & I don't feel like the article from the Immich team acknowledges it. Far too much passing the buck, not enough taking ownership.
I think my comment came across a bit harsh - the Immich team are brilliant. I've hosted it for a long time & couldn't be happier & I think my criticisms of the tone of the article are likely a case of ignorance rather than any kind of laziness or dismissiveness.
It's also in general a thankless job maintaining any open-source project, especially one of this scale, so a certain level of kneejerk cynical dismissiveness around stuff like this is expected & very forgivable.
Just really hope the ignorance / knowledge-gap can be closed off though, & perhaps some corrections to certain statements published eventually.
This may not be a huge issue depending on mitigating controls but are they saying that anyone can submit a PR (containing anything) to Immich, tag the pr with `preview` and have the contents of that PR hosted on https://pr-<num>.preview.internal.immich.cloud?
Doesn't that effectively let anyone host anything there?
I think only collaborators can add labels on github, so not quite. Does seem a bit hazardous though (you could submit a legit PR, get the label, and then commit whatever you want?).
Exposure also extends not just to the owner of the PR but anyone with write access to the branch from which it was submitted. GitHub pushes are ssh-authenticated and often automated in many workflows.
It's the result of failures across the web, really. Most browsers started using Google's phishing site index because they didn't want to maintain one themselves but wanted the phishing resistance Google Chrome has. Microsoft has SmartScreen, but that's just the same risk model but hosted on Azure.
Google's eternal vagueness is infuriating but in this case the whole setup is a disaster waiting to happen. Google's accidental fuck-up just prevented "someone hacked my server after I clicked on pr-xxxx.imiche.app" because apparently the domain's security was set up to allow for that.
You can turn off safe browsing if you don't want these warnings. Google will only stop you from visiting sites if you keep the "allow Google to stop me from visiting some sites" checkbox enabled.
I really don't know how they got nerds to think scummy advertising is cool. If you think about it, the thing they make money on - no user actually wants ads or wants to see them, ever. Somehow Google has some sort of nerd cult that people think its cool to join such an unethical company.
So unethical that they made countless free services that millions of people have relied on every day for years. Do you interface with anyone that's not deep in the software industry? Every regular person I know uses everything Google without any hesitation and no more than a bit of annoyance with ads sometimes. I think they all are pretty happy with the deal and would not switch to a paid ad-free version.
I'm increasingly blown away by takes on here that are so dramatic and militant about things that barely even register to most people.
Google's services, especially their free services, are never really free. It's just that the price tag is so well hidden that ordinary users really believe this. But the HN audience is more technical than that and they see through the smokescreen.
Except for those that are making money off adds directly or indirectly, and who believe in their god given right to my attention and my data.
> I'm increasingly blown away by takes on here that are so dramatic and militant about things that barely even register to most people.
Things 'barely even registering to most people' is not as strong a position as you may think it is. Oxygen barely registers to most people. But take it away and they register it just fine (for a short while). The 'regular' people that you know have been steadily conditioned to an ever worsening experience to the point that they barely recognize the websites they visit when seeing the web with an adblocker for the first time.
On this day, only Google Maps does not have real competitor on Android. Otherwise, it is possible to drop Google and even get better services. Brands are difficult to compete.
They created the largest spying instrument in the world that creates hidden profiles (that can never be deleted) documenting web activity, psychological state, medications, etc, etc for billions of people - and have been caught multiple times sharing data with governments (they're probably compromised internally anyway). I would categorize that as unethical. But yeah, you can cheer for the scraps they throw out.
>about things that barely even register to most people.
News flash: This whole website is about things that don't register to most people. It's called hacker news FFS.
In any case, I think a trillion dollar company probably doesn't need defending. They can easily tweak their algorithm to bury this type of stuff; after all this opinion is probably not "relevant" or "useful" to most people.
It's super simple. Check out all the Fediverse alternatives. How many people that talk a big game actually financially support those services? 2% maybe, on the high end.
Things cost money, and at a large scale, there's either capitalism, or communism.
The open internet is done. Monopolies control everything.
We have an iOS app in the store for 3 years and out of the blue apple is demanding we provide new licenses that don’t exist and threaten to kick our app out. Nothing changed in 3 years.
Getting sick of these companies able to have this level of control over everything, you can’t even self host anymore apparently.
> We have an iOS app in the store for 3 years and out of the blue apple is demanding we provide new licenses that don’t exist and threaten to kick our app out.
I write a couple of libraries for creating GOV.UK services and Google has flagged one of them as dangerous. I've appealed the decision several times but it's like screaming into a void.
I use Google Workspace for my company email, so that's the only way for me to get in contact with a human, but they refuse to go off script and won't help me contact the actual department responsible in any way.
It's now on a proper domain, https://govuk-components.x-govuk.org/ - but other than moving, there's still not much anyone can do if they're incorrectly targeted.
Us nerds *really* need to come together in creating a publicly owned browser (non chromium)
Surely among us devs, as we realize app stores increasingly hostile, that the open web is worth fighting for, and that we have the numbers to build solutions?
Firefox should be on that list. It's clearly a lot closer in functionality to Chrome/Chromium than Servo or Ladybird, so it's easier to switch to it. I like that Servo and Ladybird exist and are developing well, but there's no need to pretend that they're the only available alternatives.
A friend / client of mine used some kind of WordPress type of hosting service with a simple redirect. The host got on the bad sites list.
This also polluted their own domain, even when the redirect was removed, and had the odd side effect that Google would no longer accept email from them. We requested a review and passed it, but the email blacklist appears to be permanent. (I already checked and there are no spam problems with the domain.)
We registered a new domain. Google’s behaviour here incidentally just incentivises bulk registering throwaway domains, which doesn’t make anything any better.
I'm fighting this right now on my own domain. Google marked my family Immich instance as dangerous, essentially blocking access from Chrome to all services hosted on the same domain.
I know that I can bypass the warning, but the photo album I sent to my mother-in-law is now effectively inaccessible.
Unless I missed something in the article this seems like a different issue. The article is specifically about the domain "immich.cloud". If you're using your own domain, I'd check to ensure it hasn't been actually compromised by a bonnet or similar in some way you haven't noticed.
It may well be a false positive of Google's heuristics but home server security can be challenging - I would look at ruling out the possibility of it being real first.
It certainly sounds like a separate root issue to this article, even if the end result looks the same.
Just in case you're not sure how to deal with it, you need to request a review via the Google Search Console. You'll need a Google account and you have to verify ownership of the domain via DNS (if you want to appeal the whole domain). After that, you can log into the Google Search Console and you can find "Security Issues" under the "Security & Manual Actions" section.
That area will show you the exact URLs that got you put on the block list. You can request a review from there. They'll send you an email after they review the block.
Hopefully that'll save you from trying to hunt down non-existent malware on a half dozen self-hosted services like I ended up doing.
It's a bit ironic that a user installing immich to escape Google's grip ends up having to create again a Google account to be able to remove their Google account.
Add a custom "welcome message" in Server Settings (https://my.immich.app/admin/system-settings?isOpen=server) to make your login page look different compared to all other default Immich login pages.
This is probably the easiest non-intrusive tweak to work around the repeated flagging by Safe Browsing, still no 100% guarantee.
I agree that strict access blocking (with extra auth or IP ACL) can work better. Though I've seen in this thread https://news.ycombinator.com/item?id=45676712 and over the Internet that purely internal/private domains get flagged too. Can it be some Chrome + G Safe Browsing integration, e.g. reporting hashes of visited pages?
Looking forward to Louis Rossmann's reaction. Wouldn't be surprised if this leads to a lawsuit over monopolistic behavior - this is clearly abusing their dominant position in the browser space to eliminate competitors in photos sharing.
He's a right-to-repair activist Youtuber who is quite involved in GrayJay, another app made by this company, which is a video player client for other platforms like YouTube.
I'm not sure why his reaction would be relevant, though. It'll just be another rant about how Google has too much control like he's done in the past. He may be right, but there's nothing new to say.
He wasn't just involved with GrayJay, he's actually a member of FUTO - the company behind Immich and GrayJay. Now read grandparent comment one more time:
> Wouldn't be surprised if this leads to a lawsuit over monopolistic behavior
His reaction also matters because he's basically the public face for the company on YouTube and has a huge following. You've probably seen a bunch of social media accounts with the "clippy" character as their avatar. That's a movement started by Louis Rossman.
A good takeaway is to separate different domains for different purposes.
I had prior been tossing up the pros/cons of this (such as teaching the user to accept millions of arbitrary TLDs as official), but I think this article (and other considerations) have solidified it for me.
The biggest con of this is that to a user it will seem much more like phishing.
It happened to me a while ago that I suddenly got emails from "githubnext.com". Well, I know Github and I know that it's hosted at "github.com". So, to me, that was quite obviously phishing/spam.
This is such a difficult problem. You should be able to buy a “season pass” for $500/year or something that stops anyone from registering adjacent TLDs.
And new TLDs are coming out every day which means that I could probably go buy microsoft.anime if I wanted it.
This is what trademarks are supposed to do, but it’s reactive and not proactive.
PayPal is a real star when it comes to vague, fake-sounding, official domains.
Real users don't care much about phishing as long as you got redirected from the main domain, though. github.io has been accepted for a long time, and githubusercontent.com is invisible 99% of the time. Plus, if your regular users are not developers and still end up on your dev/staging domains, they're bound to be confused regardless.
Maybe a dumb question but what constitutes user-hosted-content?
Is a notion page, github repo, or google doc that has user submitted content that can be publicly shared also user-hosted?
IMO Google should not be able to use definitive language "Dangerous website" if its automated process is not definitive/accurate. A false flag can erode customer trust.
The definition of "active code" is broad & sometimes debatable - e.g. do old MySpace websites count - but broadly speaking the best way of thinking about it is in terms of threat model, & the main two there are:
- credential leakage
- phishing
The first is fairly narrow & pertains to uploading server side code or client javascript. If Alice hosts a login page on alice.immich.cloud that contains some session handling bugs in her code, Mallory can add some cute to mallory.immich.cloud to read cookies set on *.immich.cloud to compromise Alice's logins.
The second is much broader as it's mostly about plausible visual impersonation so will also cases where users can only upload CSS or HTML.
Specifically in this case what Immich is doing here is extremely dangerous & this post from them - while I'll give them the benefit of the doubt on being ignorant - is misinformation.
It may be dangerous but it is an established pattern. There are many cases (like Cloudflare Pages) of others doing the same, hosting strangers' sites on subdomains of a dedicated domain (pages.dev for Cloudflare, immich.cloud for Immich).
By preventing newcomers from using this pattern, Google's system is flawed, severely stifling competition.
It is but this established pattern is well standardised & documented by the public suffix list project. There's generally two conventions followed for this pattern:
1. Use a separate dedicated domain (Immich didn't do this - they're now switching to one in response to this)
2. List the separate dedicated domain in the public suffix list. As far as I can tell Immich haven't mentioned this.
This is #1 on HN for a while now and I suspect it's because many of us are nervous about it happening to us (or have already had our own homelab domains flagged!).
So is there someone from Google around who can send this along to the right team to ensure whatever heuristic has gone wrong here is fixed for good?
I doubt Google the corporation cares one bit, and any individual employees who do care would likely struggle against the system to cause significant change.
The best we all can do is to stop using Google products and encourage our friends and family to do likewise. Make sure in our own work that we don't force others to rely on Google either.
Given the scale of Google, and the nerdiness required to run Immich, I bet it's just an accident. Nevertheless, I'm very curious as to how senior Google staff looks at Immich, are they actually registering signals that people use immich-go to empty their Google Photos accounts? Do they see this as something potentially dangrous to their business in the long term?
The nerdsphere has been buzzing with Immich for some time now (I started using it a month back and it lives up to its reputation!), and I assume a lot of Googlers are in that sphere (but not neccessarily pro-Google/anti-Immich of course). So I bet they at least know of it. But do they talk about it?
I love Immich but the entire design and interface is so clearly straight up copied from Google photos. It makes me a bit nervous about their exposure, legally.
Can I use this space to comment on how amazing Immich is? I self host lots of stuff, and there’s this one tier above everything else that’s currently, and exclusively, held by Home Assistant and Immich. It is actually _better_ than Google photos (if you keep your db and thumbs on ssd, and run the top model for image search). You give up nothing, and own all your data.
This is crazy, it happened to the SoGO webmailer, standalone or bundled with the mailcow: dockerized stack as well. They implemented a slight workaround where URLs are being encrypted to avoid pattern detection to flag it as "deceiving".
There is no responses from Google about this. I had my instance flagged 3 times on 2 different domains including all subdomains, displaying a nice red banner on a representative business website. Cool stuff!
This happened to one of our documentation sites. My co-workers all saw it before I did, because Brave (my daily driver) wasn't showing it. I'm not sure if Brave is more relaxed in determining when a site is "dangerous" but I was glad not to be seeing it, because it was a false positive.
Tangential to the flagging issue, but is there any documentation on how Immich is doing the PR site generation feature? That seems pretty cool, and I'd be curious to learn more.
Sometimes it is also rude to ask without looking the obvious place themselves. It is about signaling that ”my” time is more precious than ”your” time so I let them do that check for me, if I can use someone elses time.
I think we might have hit the inflection point where being rude is more polite. It's not that I want people to be rude to me, it's that I don't want to talk to AI when I intend to be talking to a person, and anyone engaging with me via AI is infinitely more disrespectful than any curse word or rudeness.
These days, when I get a capitalized, grammatically correct sentence — and proper punctuation to boot, there is an unfortunate chance it was written using an AI and I am not engaging fully with a human.
its when my covnersation partner makes human mistakes, like not capitalizing things, or when they tell me i'm a bonehead, that i know i'm talking to a real human not a bot. it makes me feel happier and more respected. i want to interact with humans dammit, and at this point rude people are more likely to be human than polite ones on the internet.
i know you can prompt AIs to make releaistic mistakes too, the arms race truly never ends
Pretty sure Immich is on github, so I assume they have a workflow for it, but in case you're interested in this concept in general, gitlab has first-class support for this which I've been using for years: https://docs.gitlab.com/ci/review_apps/ . Very cool and handy stuff.
Safe Browsing collects a lot of data, such as hashes of URLs (URLs can be easily decoded by comparison) and probably other interactions with web like downloads.
But how effective is it in malware detection?
The benefits seem to me dubious. It looks like a feature offered to collect browsing data, useful to maybe 1% in special situations.
I’ve heard anecdotes of people using an entirely internal domain like “plex.example.com” even if it’s never exposed to the public internet, google might flag it as impersonating plex. Google will sometimes block it based only on name, if they think the name is impersonating another service.
Its unclear exactly what conditions cause a site to get blocked by safe browsing. My nextcloud.something.tld domain has never been flagged, but I’ve seen support threads of other people having issues and the domain name is the best guess.
I'm almost positive GMail scanning messages is one cause. My domain got put on the list for a URL that would have been unknowable to anyone but GMail and my sister who I invited to a shared Immich album. It was a URL like this that got emailed directly to 1 person:
Then suddenly the domain is banned even though there was never a way to discover that URL besides GMail scanning messages. In my case, the server is public so my siblings can access it, but there's nothing stopping Google from banning domains for internal sites that show up in emails they wrongly classify as phishing.
Think of how Google and Microsoft destroyed self hosted email with their spam filters. Now imagine that happening to all self hosted services via abuse of the safe browsing block lists.
if it was just the domain, remember that there is a Cert Transparency log for all TLS certs issued nowadays by valid CAs, which is probably what Google is also using to discover new active domains
It doesn’t seem like email scanning is necessary to explain this. It appears that simply having a “bad” subdomain can trigger this. Obviously this heuristic isn’t working well, but you can see the naive logic of it: anything with the subdomain “apple” might be trying to impersonate Apple, so let’s flag it. This has happened to me on internal domains on my home network that I've exposed to no one. This also has been reported at the jellyfin project: https://github.com/jellyfin/jellyfin-web/issues/4076
That's not going to be gleaned from a CT log or guessed randomly. The URL was only transmitted once to one person via e-mail. The sending was done via MXRoute and the recipient was using GMail (legacy Workspace).
The only possible way for Google to have gotten that URL to start the process would have been by scanning the recipient's e-mail.
Well, that's potentially horrifying. I would love for someone to attempt this in as controlled of a manner as possible. I would assume it's possible for anyone using Google DNS servers to also trigger some type of metadata inspection resulting in this type of situation as well.
Also - when you say banned, you're speaking of the "red screen of death" right? Not a broader ban from the domain using Google Workplace services, yeah?
> Also - when you say banned, you're speaking of the "red screen of death" right?
Yes.
> I would love for someone to attempt this in as controlled of a manner as possible.
I'm pretty confident they scanned a URL in GMail to trigger the blocking of my domain. If they've done something as stupid as tying GMail phishing detection heuristics into the safe browsing block list, you might be able to generate a bunch of phishy looking emails with direct links to someone's login page to trigger the "red screen of death".
This reminds me of another post where a scammer sent a gmail message containing https://site.google.com/xxx link to trick users into click, but gmail didn't detect the risk.
I'm kind of curious, do you have your own domain for immich or is this part of a malware-flagged subdomain issue? It's kind of wild to me that Google would flag all instances of a particular piece of self-hosted software as malicious.
- A self-hosted project has a demo instance with a default login page (demo.immich.app, demo.jellyfin.org, demo1.nextcloud.com) that is classified as "primary" by google's algorithms
- Any self-hosted instance with the same login page (branding, title, logo, meta html) becomes a candidate for deceptive/phishing by their algorithm. And immich.cloud has a lot of preview envs falling in that category.
BUT in Immich case its _demo_ login page has its own big banner, so it is already quite different from others.
Maybe there's no "original" at all. The algorithm/AI just got lost among thousands of identically looking login pages and now considers every other instance as deceptive...
Google often marks my homelab domains as dangerous which all point to an A record that is in the private IP space, completely inaccessible to the internet.
> There is a user in the JavaScript community who goes around adding "backwards compatibility" to projects. They do this by adding 50 extra package dependencies to your project, which are maintained by them.
Not sure if this is exactly the scenario from the discussed article but it's interesting to understand it nonetheless.
TL;DR the browser regularly downloads a dump of color profile fingerprints of known bad websites. Then when you load whatever website, it calculates the color profile fingerprint of it as well, and looks for matches.
(This could be outdated and there are probably many other signals.)
I had my personal domain I use for self-hosting flagged. I've had the domain for 25 years and it's never had a hint of spam, phishing, or even unintentional issues like compromised sites / services.
It's impossible to know what Google's black box is doing, but, in my case, I suspect my flagging was the result of failing to use a large email provider. I use MXRoute for locally hosted services and network devices because they do a better job of giving me simple, hard limits for sending accounts. That way if anything I have ever gets compromised, the damage in terms of spam will be limited to (ex) 10 messages every 24h.
I invited my sister to a shared Immich album a couple days ago, so I'm guessing that GMail scanned the email notifying her, used the contents + some kind of not-google-or-microsoft sender penalty, and flagged the message as potential spam or phishing. From there, I'd assume the linked domain gets pushed into another system that eventually decides they should blacklist the whole domain.
The thing that really pisses me off is that I just received an email in reply to my request for review and the whole thing is a gas-lighting extravaganza. Google systems indicate your domain no longer contains harmful links or downloads. Keep yourself safe in the future by blah blah blah blah.
Umm. No! It's actually Google's crappy, non-deterministic, careless detection that's flagging my legitimate resources as malicious. Then I have to spend my time running it down and double checking everything before submitting a request to have the false positive mistake on Google's end fixed.
Convince me that Google won't abuse this to make self hosting unbearable.
> I suspect my flagging was the result of failing to use a large email provider.
This seems like the flagging was a result of the same login page detection that the Immich blog post is referencing? What makes you think it's tied to self-hosted email?
I'm not using self hosted email. My theory is that Google treats smaller mail providers as less trustworthy and that increases the odds of having messages flagged for phishing.
In my case, the Google Search Console explicitly listed the exact URL for a newly created shared album as the cause.
I wish I would have taken a screenshot. That URL is not going to be guessed randomly and the URL was only transmitted once to one person via e-mail. The sending was done via MXRoute and the recipient was using GMail (legacy Workspace).
The only possible way for Google to have gotten that URL to start the process would have been by scanning the recipient's e-mail. What I was trying to say is that the only way it makes sense to me is if Google via GMail categorized that email as phishing and that kicked off the process to add my domain to the block list.
So, if email categorization / filtering is being used as a heuristic for discovering URLs for the block list, it's possible Google's discriminating against domains that use smaller email hosts that Google doesn't trust as much as themselves, Microsoft, etc..
All around it sucks and Google shouldn't be allowed to use non-deterministic guesswork to put domains on a block list that has a significant negative impact. If they want to operate a clown show like that, they should at least be liable for the outcomes IMO.
I'm in a similar boat. Google's false flag is causing issues for my family members who use Chrome, even for internal services that aren't publicly exposed, just because they're on related subdomains.
It's scary how much control Google has over which content people can access on the web - or even on their local network!
I've rarely seen a HN comment section this overwhelmingly wrong on a technical topic. This community is usually better than this.
Google is an evil company I want the web to be free of, I resent that even Firefox & Safari use this safe browsing service. Immich is a phenomenal piece of software - I've hosted it myself & sung its praises on HN in the past.
Put putting aside David vs Goliath biases here, Google is 100% correct here & what Immich are doing is extremely dangerous. The fact they don't acknowledge that in the blog post shows a security knowledge gap that I'm really hoping is closed over the course of remediating this.
I don't think the Immich team mean any harm but as it currently stands the OP constitutes misinformation.
They're auto-deploying PRs to a subdomain of a domain that they also use for production traffic. This allows any member of the public with a GitHub account to deploy any arbitrary code to that subdomain without any review or approval from the Immich team. That's bad for two reasons:
1. PR deploys on public repos are inherently tricky as code gains access to the server environment, so you need to be diligent about segregating secrets for pr deployments from production secret management. That diligence is a complex & continuous undertaking, especially for an open source project.
2. Anyone with a GitHub account can use your domain for phishing scams or impersonation.
The second issue is why they're flagged by Google (he first issue may be higher risk to the Immich project but it's out of scope for Google's safe browsing service).
To be clear: this isn't about people running their own immich instance. This is about members of the public having the ability to deploy arbitrary code without review.
---
The article from the Immich team does mention they're switching to using a non-production domain (immich.build) for their PR builds which does indicate to me they somewhat understand the issue (though they've explained it badly in the article), but they don't seem to understand the significance or scope.
> This allows any member of the public with a GitHub account to deploy any arbitrary code to that subdomain without any review or approval from the Immich team.
This part is not correct: the "preview" label can be set only by collaborators.
> a subdomain of a domain that they also use for production traffic
To clarify this part: the only production traffic that immich.cloud serves are static map tiles (tiles.immich.cloud)
Overall, I share your concerns, and as you already mentioned, a dedicated "immich.build" domain is the way to go.
> This part is not correct: the "preview" label can be set only by collaborators.
That's good & is a decent starting point. A decent second step might be to have the Github Actions workflow also check the approval status of the PR before deploying (requiring all collaborators to be constantly aware that the risk of applying a label is similar to that of an approval seems less viable)
I've had it work for me several times. Most of the time following links/redirects from search engines, ironically a few times from Google itself. Not that I was going to enter anything (the phishing attempts themselves were quite amateurish) but they do help in some rare cases.
When I worked customer service, these phishing blocks worked wonders preventing people from logging in to your-secure-webmail.jobz. People would be filling in phishing forms days after sending out warnings on all official channels. Once Google's algorithm kicked in, the attackers finally needed to switch domains and re-do their phishing attempts.
Honestly, where do people live that the DMV (or equivalent - in some states it is split or otherwise named) is a pain? Every time I've ever been it has been "show up, take a number, wait 5 minutes, get served" - and that's assuming website self-service doesn't suffice.
I’m also self hosting gitea and pertainer and I’m trying this issue every few weeks. I appeal, they remove the warning, after a week is back. This is ongoing for at least 4 years. I have more than 20 appeals all successfully removing the warning. Ridiculous. I heard legal action is the best option now, any other ideas?
First thing I do when I start to use a browser for the first time is making sure 'Google Safe Browsing' feature is disabled. I don't need yet another annoyance while I browse the web, especially when it's from Google.
I think the other very interesting thing in the reddit thread[0] for this is that if you do well-known-domain.yourdomain.tld then you're likely to get whacked by this too. It makes sense I guess. Lots of people are probably clicking gmail.shady.info and getting phished.
> The most alarming thing was realizing that a single flagged subdomain would apparently invalidate the entire domain.
Correct. It works this way because in general the domain has the rights over routing all the subdomains. Which means if you were a spammer, and doing something untoward on a subdomain only invalidated the subdomain, it would be the easiest game in the world to play.
I’d say this is a clear slight from Google, using their Chrome browser because something or someone is inconveniencing another part of their business, google cloud / google photos.
They did a similar thing with the uBlock Origin extension, flagging it with “this extension might be slowing down your browser” in a big red banner in the last few months of manifest v2 on Chrome. After already having to upload the extension yourself to Chrome cause they took it off the extension store cause it was inhibiting on their ad business.
Google is a massive monopolistic company who will pull strings on one side of their business to help another.
With only Firefox not being based on Chromium and still having manifest v2 the future (5 to 10 years from now) looks bleak. With only 1 browser like this web devs can phase it out slowly by not taking it into consideration when coding or Firefox could enshittify to such an extent because of their manifest v2 monopoly that even that wont make it worth it anymore.
Oh and for the ones not in the know, Manifest is the name of a javascript file manifest.js that decides what browser extensions can and cant modify and the “upgrade” from manifest v2 to v3 has made it near impossible for adblockers to block ads.
If there are any googlers here, I'd like to report an even more dangerous website. As much as 30-50% of the traffic to it relates to malware or scams, and it has gone unpunished for a very long time.
The address appears to be adsense.google.com.
What i really don't understand at least here in Europe the advertising partner (adsense) must investigate at least minimally whether the advertising is illegal or fraudulent, i understand that sites.google etc are under "safe harbor" but that's not the point with adsense since people from google "click" the publish button and also get money to publish that ad.
The law is only for plebs like you and me. Companies get a pass.
I'm still amazed how deploying spyware would've rightfully landed you in jail a couple decades back, but do the same thing on the web under the justification of advertising/marketing and suddenly it's ok.
>Companies get a pass.
I'm pretty sure that if Springer were to make a fraudulent ad, they would instantly be slapped with a lawsuit and face public outcry.
Springer itself is nothing but scam.
True, but at least the ad's are not ;)
Also YouTube.com serves a lot of scam advertisements. They should block that too.
I think google is crumbling under the weight of their size. They are no longer able to process the requested commercials with due diligence.
Nah, they just don't give a fuck. Never have
I see the same scam/deepfake ad(s) pretty much persistently. Maybe they actually differ slightly (they are AI gen mostly), but it's pretty obvious what they are, and I'm sure they get flagged a lot.
They just need to introduce a basic deposit to post ads, and you lose it if you put up a scam ad. Would soon pay for the staff needed to police it, and prevent scammers from bypassing admin by trivially creating new accounts.
Yeah - that website keeps on spamming me down with useless stuff.
I was able to block most of this via ublock origin but Google disabled this - can not download it from here anymore:
https://chromewebstore.google.com/detail/ublock-origin/cjpal...
Funniest nonsense "explanation":
"This extension is no longer available because it doesn't follow best practices for Chrome extensions."
In reality Google killed it because it threatens their greed income. Ads, ads and more ads.
You know what? I don't even mind them killing it, because of course there are a whole pile of items under the anti-trust label that google is doing so why not one more. But what I do take issue with is the gaslighting, their attempt to make the users believe that this is in the users interests, rather than in google's interests.
If we had functional anti-trust laws then this company would have been broken up long ago, Alphabet or not. But they keep doing these things because we - collectively - let them.
Why would a monopoly care about users interests?
I know they won't. But we have all the tools to force them to care. We just don't use the tools effectively, and between that and lobbying they get a free pass to pretty much do as they please.
Use Firefox.
Use one of the forks. librewolf, waterfox, zen. Firefox itself lost trust when Mozilla tried to push the new Terms of Use earlier this year. That was so aggressively user-hostile that nobody should trust Mozilla ever again. Using a fork puts an insulation layer between you and Mozilla.
Librewolf is just a directly de-mozillaed and privacy-enhanced Firefox, similar to Ungoogled Chromium. I've been trying to get in the habit of using Zen Browser, which has a bunch of UI changes.
> Firefox itself lost trust when Mozilla tried to push the new Terms of Use earlier this year.
Those terms of use aren't in place any longer. I'm surprised that listening to the users is viewed as something bad.
The problem is that all those forks are beholden to Mozilla's corporate interests the same way the chromium derivatives are beholden to Google's corporate interests. What we need is one of the newer independent engines to mature - libweb, servo or blitz.
How are they beholden? In the sense that it's hard to provide engine updates without the funding of goog?
edit: also, by "libweb", did you mean "ladybird"?
Apparently the "best practise" is using Manifest V3 versus V2.
Reading a bit online (not having any personal/deep knowledge) it seems the original extension also downloaded updates from a private (the developers) server, while that is no longer allowed - they now need to update via the chrome extension, which also means waiting for code review/approval from google.
I can see the security angle there, it is just awkward how much of an vested interest google has in the whole topic. ad-blocking is already a grey area (legally), and there is a cat-and-mouse between blockers and advertisers; it's hard to believe there is only security best-practise going on here.
sites.google.com
The same outfit is runimg a domain called blogger.
Reminds me of MS blocking a website of mine for dangerous script. The offending thing i did was use document.write to put copyright 2025 (with the current year) at the end of static pages.
My work's email filter regularly flags links to JIRA and github as dangerous. It stopped being even ironically amusing after a while.
Microsoft's own Outlook.com flags Windows Insider emails coming from a .microsoft.com domain as junk even after marking the domain as "no junk". They know themselves well.
Frequent frustration past week for me:
The integrated button to join a Microsoft Teams meeting directly from my Microsoft Outlook Calendar doesn't work because Microsoft needs to scan the link from Microsoft to Microsoft for malware before proceeding, and the malware scanning service has temporary downtime and serves me static page saying "The content you are accessing cannot currently be verified".
I feel like the GitHub one might be okay since a lot of malware binaries are hosted there still.
sites.google.com is widely abused but so practically any site which allows users to host content of their choice and make it publicly available. Where google can be different is that they famously refuse yo do work which they cannot automate and probably they cannot (or don’t want) to automate detection/blocking of spam/phishing hosted on sites.google.com and processing of abuse reports.
The nerve of letting everyone run a phishing campaign on sites.google.com but marking a perfectly safe website as malicious.
Enshitification ensues.
Happened to me last week. One morning we wake up and the whole company website does not work.
Not advice with some time to fix any possible problem, just blocked.
We gave very bad image to our clients and users, and had to give explanations of a false positive from google detection.
The culprit, according to google search console, was a double redirect on our web email domain (/ -> inbox -> login).
After just moving the webmail to another domain, removing one of the redirections just in case, and asking politely 4 times to be unblocked.. took about 12 hours. And no real recourse, feedback or anything about when its gonna be solved. And no responsibility.
The worse is the feeling of not in control of your own business, and depending on a third party which is not related at all with us, which made a huge mistake, to let out clients use our platform.
I'm beginning to seriously think we need a new internet, another protocol, other browsers just to break up the insane monopolies that has been formed, because the way things are going soon all discourse will be censored, and competitors will be blocked soon.
We need something that's good for small and medium businesses again, local news and get an actual marketplace going - you know what the internet actually promised.
Anyone working on something like this?
It's very, very hard to overcome the gravitational forces which encourage centralization, and doing so requires rooting the different communities that you want to exist in their own different communities of people. It's a political governance problem, not a technical one.
The community around NOSTR are basically building a kind of semantic web, where users identities are verified via their public key, data is routed through content agnostic relays, and trustworthiness is verified by peer recommendation.
They are currently experimenting with replicating many types of services which are currently websites as protocols with data types, with the goal being that all of these services can share available data with eachother openly.
It's definitely more of a "bazaar" model over a "catherdral" model, with many open questions and it's also tough to get a good overview of what is really going on there. But at least it's an attempt.
It will take the same or less amount of time, to get where we are with current Web.
What we have is the best sim env to see how stuff shape up. So fixing it should be the aim, avoiding will get us on similar spirals. We'll just go on circles.
Having a decade of fresh air is also a good incentive regardless of how it ends
It won't get anywhere unless it addresses the issue of spam, scammers, phishing etc. The whole purpose of Google Safe Browsing is to make life harder for scammers.
True, but google already censors their search results to push certain imperial agendas so i'm not trusting them in the long run.
I'm afraid this can't be built on the current net topology which is owned by the Stupid Money Govporation and inherently allows for roadblocks in the flow of information. Only a mesh could solve that.
But the Stupid Money Govporation must be dethroned first, and I honestly don't see how that could happen without the help of an ELE like a good asteroid impact.
If you're going to host user content on subdomains, then you should probably have your site on the Public Suffix List https://publicsuffix.org/list/ . That should eventually make its way into various services so they know that a tainted subdomain doesn't taint the entire site....
God I hate the web. The engineering equivalent of a car made of duct tape.
> Since there was and remains no algorithmic method of finding the highest level at which a domain may be registered for a particular top-level domain
A centralized list like this not just for domains as a whole (e.g. co.uk) but also specific sites (e.g. s3-object-lambda.eu-west-1.amazonaws.com) is both kind of crazy in that the list will bloat a lot over the years, as well as a security risk for any platform that needs this functionality but would prefer not to leak any details publicly.
We already have the concept of a .well-known directory that you can use, when talking to a specific site. Similarly, we know how you can nest subdomains, like c.b.a.x, and it's more or less certain that you can't create a subdomain b without the involvement of a, so it should be possible to walk the chain.
Example:
Maybe ship the domains with the browsers and such and leave generic sites like AWS or whatever to describe things themselves. Hell, maybe that could also have been a TXT record in DNS as well.> any platform that needs this functionality but would prefer not to leak any details publicly.
I’m not sure how you’d have this - it’s for the public facing side of user hosted content, surely that must be public?
> We already have the concept of a .well-known directory that you can use, when talking to a specific site.
But the point is to help identify dangerous sites, by definition you can’t just let the sites mark themselves as trustworthy and rotate around subdomains. If you have an approach that doesn’t have to trust the site, you also don’t need any definition at the top level you could just infer it.
I presume it has to be a curated list otherwise spammers would use it to evade blocks. Otherwise why not just use DNS?
Whois would be the choice. DNS’s less glamourous sibling, purpose built for delegated publication of accountability records
> God I hate the web
This is mostly a browser security mistake but also partly a product of ICANN policy & the design of the domain system, so it's not just the web.
Also, the list isn't really that long, compared to, say, certificate transparency logs; now that's a truly mad solution.
Show me a platform not made out of duct tape and I'll show you a platform nobody uses.
regular cars?
The Honda issue where setting a certain radio station, would brick the infotainment? That good enough?
> That good enough?
Not really. Does the car still drive? That sounds like a software bug; hardly indicative that the entire car is held together with duct tape, but a pretty bad bug non the less.
Never heard of this. Link please?
Don't know about Honda, but there is this Mazda one [0] (Would not be surprised if it affected multiple vendors!)
[0] https://www.soundandvision.com/content/remembering-time-when...
Admitting I'm old, but my HP-11C still gets pretty-regular use.
And judging by eBay prices, or the SwissMicros product line, I suspect I have plenty of company.
I think we lost the web somewhere between PageRank and JavaScript. Up to there it was just linked documents and it was mostly fine.
"The engineering equivalent of a car made of duct tape"
Kind of. But do you have a better proposition?
I'd probably say we ought to use DNS.
And while we’re at it, 1) mark domains as https-only, and 2) when root domains map to a subdomain (eg www).
A part of the issue is IMO that browsers have become ridiculously bloated everything-programs. You could take about 90% of that out and into dedicated tools and end up with something vastly saner and safer and not a lot less capable for all practical purposes. Instead, we collectively are OK with frosting this atrocious layer cake that is today's web with multiple flavors of security measures of sometimes questionable utility.
End of random rant.
"You could take about 90% of that out and into dedicated tools "
But then you would loose plattform independency, the main selling point of this atrocity.
Having all those APIs in a sandbox that mostly just work on billion devices is pretty powerful and a potential succesor to HTML would have to beat that, to be adopted.
The best thing to happen, that I can see, is that a sane subset crystalizes, that people start to use dominantly, with the rest becoming legacy, only maintained to have it still working.
But I do dream of a fresh rewrite of the web since university (and the web was way slimmer back then), but I got a bit more pragmatic and I think I understood now the massive problem of solving trusted human communication better. It ain't easy in the real world.
But do we need e.g serial port or raw USB access straight from a random website? Even WebRTC is a bit of a stretch. There is a lot of cruft in modern browsers that does little except increase attack surface.
This all just drives a need to come up with ever more tacked-on protection schemes because browsers have big targets painted on them.
Itch.io games and controller support.
You have sites now that let you debug microcontrollers on your browser, super cool.
Same thing but with firmware updates in the browser. Cross platform, replaced a mess of ugly broken vendor tools.
WebRTC I use since many years and would miss it a lot. P2P is awesome.
WebUSB I don't use or would miss it right now, but .. the main potential use case is security and it sounds somewhat reasonable
"Use in multi-factor authentication
WebUSB in combination with special purpose devices and public identification registries can be used as key piece in an infrastructure scale solution to digital identity on the internet."
https://en.wikipedia.org/wiki/WebUSB
> Even WebRTC is a bit of a stretch
You remove that, and videoconferencing (for business or person to person) has to rely on downloading an app, meaning whoever is behind the website has to release for 10-15 OSes now. Some already do, but not everyone has that budget so now there's a massive moat around it.
> But do we need e.g serial port or raw USB access straight from a random website
Being able to flash an IoT (e.g. ESP32) device from the browser is useful for a lot of people. For the "normies", there was also Stadia allowing you to flash their controller to be a generic Bluetooth/usb one on a website, using that webUSB. Without it Google would have had to release an app for multiple OSes, or more likely, would have just left the devices as paperweights. Also, you can use FIDO/U2F keys directly now, which is pretty good.
Browsers are the modern Excel, people complain that they do too much and you only need 20%. But it's a different 20% for everyone.
> But do we need e.g serial port or raw USB access straight from a random website?
Yes. Regards, CIA, Mossad, FSB etc.
How else am I going to make a game in the browser that be controlled with a controller?
Every decent host OS already has a dedicated driver stack to provide game controller input to applications in a useful manner. Why the heck would you ship a reimplementation of that in JS in a website?
So that you can take input from countries that haven't been invented yet and won't fit the HID model.
Not sure if it counts but I've been enjoying librewolf. I believe just a stripped down firefox.
> Having all those APIs in a sandbox that mostly just work on billion devices is pretty powerful and a potential succesor to HTML would have to beat that, to be adopted.
I think the giant major downside, is that they've written a rootkit that runs on everything, and to try to make up for that they want to make it so only sites they allow can run.
It's not really very powerful at all if nobody can use it, at that point you are better off just not bothering with it at all.
The Internet may remain, but the Web may really be dead.
"It's not really very powerful at all if nobody can use it"
But people do use it, like the both of us right now?
People also use maps, do online banking, play games, start complex interactive learning environments, collaborate in real time on documents etc.
All of that works right now.
> to try to make up for that they want to make it so only sites they allow can run
What do you mean, you can run whatever you want on localhost, and it's quite easy to host whatever you want for whoever you want too. Maybe the biggest modern added barrier to entry is that having TLS is strongly encouraged/even needed for some things, but this is an easily solved problem.
>A part of the issue is IMO that browsers have become ridiculously bloated everything-programs.
I don't see how that solves the issue that PSL tries to fix. I was a script kiddy hosting neopets phishing pages on free cpanel servers from <random>.ripway.com back in 2007. Browsers were way less capable then.
PSL and the way cookies work is just part of the mess. A new approach could solve that in a different way, taking into account all the experience we had with scriptkiddies and professional scammers and pishers since then. But I also don't really have an idea where and how to start.
And of course, if the new solution completely invalidates old sites, it just won't get picked up. People prefer slightly broken but accessible to better designed but inaccessible.
> People prefer slightly broken but accessible to better designed but inaccessible.
We live in world where whatever faang adopts is de facto a standard. Accessible these days means google/gmail/facebook/instagram/tiktok works. Everything else is usually forced to follow along.
People will adopt whatever gives them access to their daily dose of doomscrolling and then complain about rather crucial part of their lives like online banking not working.
> And of course, if the new solution completely invalidates old sites, it just won't get picked up.
Old sites don't matter, only high-traffic sites riddled with dark patterns matter. That's the reality, even if it is harsh.
> People prefer slightly broken but accessible to better designed but inaccessible.
It's not even broken as the edge cases are addressed by ad-hoc solutions.
OP is complaining about global infrastructure not having a pristine design. At best it's a complain over a desirable trait. It's hardly a reason to pull the Jr developer card and mindlessly advocate for throwing everything out and starting over.
Are you saying we should make a <Unix Equivalent Of A Browser?> A large set of really simple tools that each do one thing really really really pedantically well?
This might be what's needed to break out of the current local optimum.
Maybe it's time to revive something like the uzbl[1] project, or start something similar.
[1] https://www.uzbl.org/
I haven't thought of it that way, but that might be a solution.
There was an attempt in that direction.
https://www.uzbl.org/
You are right from a technical point, I think, but in reality - how would one begin to make that change?
Why is it a centrally maintained list of domains, when there is a whole extensible system for attaching metadata to domain names?
That's the nature of decentralised control. It's not just DNS, phone numbers work in the same way.
I think it's somewhat tribal webdev knowledge that if you host user generated content you need to be on the PSL otherwise you'll eventually end up where Immich is now.
I'm not sure how people not already having hit this very issue before is supposed to know about it beforehand though, one of those things that you don't really come across until you're hit by it.
This is the first time I hear about https://publicsuffix.org
You're in good company! From 12 days ago: https://news.ycombinator.com/item?id=45538760
I’ve been doing this for at least 15 years and it’s the first I heard of this.
Fun learning new things so often but I never once heard of the public suffix list.
That said, I do know the other best practices mentioned elsewhere
First rule of the public suffix list...
so its skill issue ??? or just google being bad????
I will go with Google being bad / evil for 500.
Google 90s to 2010 is nothings like Google 2025. There is a reason they removed "Don't be evil" ... being evil and authoritarian makes more money.
Looking at you Manifest V2 ... pour one out for your homies.
Don't get me wrong, Google is bad/evil in many ways, but the public suffix list exists to solve a real risk to users. Google is flagging this for a legit reason in this particular case.
Sympathy for the devil, people keep using Google's browser because the safe search guards catch more bad actors than they false positive good actors.
> people keep using Google's browser because the safe search guards catch more bad actors than they false positive good actors.
This is the first thing i disable in Chrome, Firefox and Edge. The only safe thing they do is safely sending all my browsing history to Google or Microsoft.
downvoted for saying truth
many google employee is in here, so I dont expect them to be agree with you
Looking through some of the links in this post, I there are actually two separate issues here:
1. Immich hosts user content on their domain. And should thus be on the public suffic list.
2. When users host an open source self hosted project like immich, jellyfin, etc. on their own domain it gets flagged as phishing because it looks an awful lot like the publicly hosted version, but it's on a different domain, and possibly a domain that might look suspicious to someone unfamiliar with the project, because it includes the name of the software in the domain. Something like immich.example.com.
The first one is fairly straightforward to deal with, if you know about the public suffix list. I don't know of a good solution for the second though.
I don't think the Internet should be run by being on special lists (other than like, a globally run registry of domain names)...
I get that SPAM, etc., are an issue, but, like f* google-chrome, I want to browse the web, not some carefully curated list of sites some giant tech company has chosen.
A) you shouldn't be using google-chrome at all B) Firefox should definitely not be using that list either C) if you are going to have a "safe sites" list, that should definitely be a non-profit running that, not an automated robot working for a large probably-evil company...
> I don't think the Internet should be run by being on special lists
People are reacting as if this list is some kind of overbearing way of tracking what people do on the web - it's almost the opposite of that. It's worth clarifying this is just a suffix list for user-hosted content. It's neither a list of user-hosted domains nor a list of safe websites generally - it's just suffixes for a very small specific use-case: a company providing subdomains. You can think of this as a registry of domain sub-letters.
For instance:
- GitHub.io is on the list but GitHub.com is not - GitHub.com is still considered safe
- I self-host an immich instance on my own domain name - my immich instance isn't flagged & I don't need to add anything to the list because I fully own the domain.
The specific instance is just for Immich themselves who fully own "immich.cloud" but sublet subdomains under it to users.
> *if you are going to have a "safe sites" list"
This is not a safe sites list! This is not even a sites list at all - suffixes are not sites. This also isn't even a "safe" list - in fact it's really a "dangerous" list for browsers & various tooling to effectively segregate security & privacy contexts.
Google is flagging the Immich domain not because it's missing from the safe list but because it has legitimate dangers & it's missing from the dangerous list that informs web clients of said dangers so they can handle them appropriately.
It always has been run on special lists.
I've coined the phrase "Postel decentralization" to refer to things where people expect there to be some distributed consensus mechanism but it turned out that the design of the internet was to email Jon Postel (https://en.wikipedia.org/wiki/Jon_Postel) to get your name on a list. e.g. how IANA was originally created.
Firefox and Safari also use the list. At least by default, I think you can turn it off in firefox. And on the whole, I think it is valuable to have _a_ list of known-unsafe sites. And note that Safe Browsing is a blocklist, not an allowlist.
The problem is that at least some of the people maintaining this list seem to be a little trigger happy. And I definitely thing Google probably isn't the best custodian of such a list, as they have obvious conflicts of interest.
>I think it is valuable to have _a_ list of known-unsafe sites
And how and who should define what is consider unsafe sites?
Ideally there should be several/many and the user should be able to direct their browser as to which they would like to use (or none at all)
Oh god, you reminded me the horrors of hosting my own mailserver and all of the white/blacklist BS you have to worry about being a small operator (it's SUPER easy to end up on the blacklists, and is SUPER hard to get onto whitelists)
You can turn it off in Chrome settings if you want.
If you have such strong feelings, you could always use vanilla chromium.
There are other browsers if you want to browse the web with the blinders off.
It's browser beware when you do, but you can do it.
> I don't know of a good solution for the second though.
I know the second issue can be a legitimate problem but I feel like the first issue is the primary problem here & the "solution" to the second issue is a remedy that's worse than the disease.
The public suffix list is a great system (despite getting serious backlash here in HN comments, mainly from people who have jumped to wildly exaggerated conclusions about what it is). Beyond that though, flagging domains for phishing for having duplicate content smells like an anti-self-host policy: sure there's phishers making clone sites, but the vast majority of sites flagged are going to be legit unless you employ a more targeted heuristic, but doing so isn't incentivised by Google's (or most company's) business model.
The second is a real problem even with completely unique applications. If they have UI portions that have lookalikes, you will get flagged. At work, I created an application with a sign-in popup. Because it's for internal use only, the form in the popup is very basic, just username and password and a button. Safe Browsing continues to block this application to this day, despite multiple appeals.
> When users host an open source self hosted project like immich, jellyfin, etc. on their own domain...
I was just deploying your_spotify and gave it your-spotify.<my services domain> and there was a warning in the logs that talked about thud, linking the issue:
https://github.com/Yooooomi/your_spotify/issues/271
Even the first one only works if there's no need to have site-wide user authentication on the domain, because you can't have a domain cookie accessible from subdomains anymore otherwise.
That means the Safe Browsing abuse could be weaponized against self-hosted services, oh my...
New directive from the Whitehouse. Block all non approved sites. If you don't do it we will block your merger etc...
The issue isn't the user-hosted content - I'm running a release build of Immich on my own server and Google flagged my entire domain.
Is it on your own domain?
Yes, my own domain.
They aren't hosting user content; it was their pull request preview domains that was triggering it.
This is very clearly just bad code from Google.
I thought this story would be about some malicious PR that convinced their CI to build a page featuring phishing, malware, porn, etc. It looks like Google is simply flagging their legit, self-created Preview builds as being phishing, and banning the entire domain. Getting immich.cloud on the PSL is probably the right thing to do for other reasons, and may decrease the blast radius here.
The root cause is bad behaviour by google. This is merely a workaround.
Remember, this is a free service that Google is offering for even their competitors to use.
And it is incredibly valuable thing. You might not think it is, but internet is filled utterly dangerous, scammy, phisy, malwary websites and everyday Safe Browsing (via Chrome, Firefox and Safari - yes, Safari uses Safe Browsing) keeps users safe.
If immich didnt follow best practice that's Google's fault? You're showing your naivety, and bias here.
Please point me to where GoDaddy or any other hosting site mentions public suffix, or where Apple or Google or Mozilla have a listing hosting best practices that include avoiding false positives by Safe Browsing…
>GoDaddy or any other hosting site mentions public suffix
They don't need to mention it because they handle it on behalf of the client. Them recommending best practices like using separate domains makes as much sense as them recommending what TLS configs to use.
>or where Apple or Google or Mozilla have a listing hosting best practices that include avoiding false positives by Safe Browsing…
Since were those sites the go to place to learn how to host a site? Apple doesn't offer anything related to web hosting besides "a computer that can run nginx". Google might be the place to ask if you were your aunt and "google" means "internet" to her. Mozilla is the most plausible one because they host MDN, but hosting documentation on HTML/CSS/JS doesn't necessarily mean they offer hosting advice, any more than expecting docs.djangoproject.com to contain hosting advice.
The underlying question is how are people supposed to know about this before they have a big problem?
If you have a service where anyone can sign up and host content on your subdomain, it really is your responsibility to know. Calling this "unfair" because you didn't know is naive.
If amazon shutdown your AWS account, because those same scammers used those domains to host CP rather than phishing pages, would you accept the excuse of "how was I supposed to know?"
Nothing in this article indicates UGC is the problem. It's that Google thinks there's an "official" central immich and these instances are impersonating it.
What malicious UGC would you even deliver over this domain? An image with scam instructiins? CSAM isn't even in scope for Safe Browsing, just phishing and malware.
>You might not think it is, but internet is filled utterly dangerous, scammy, phisy, malwary websites
Google is happy to take their money and show scammy ads. Google ads are the most common vector for fake software support scams. Most people google something like "microsoft support" and end up there. Has Google ever banned their own ad domains?
Google is the last entity I would trust to be neutral here.
Holy shit look into the mirror.
One of the internet's biggest source of scams, phishing, and malware and everything you are complaining about is google adsense.
Google is using the list to bully out competitors, while telling you it's for keeping you safe.
_You_ are showing naivety and bias.
You should not be downvoted. Either HN has had an influx of ignorant normies or it's google bots attacking any negative comments
Oh c’mon. Google does not offer free services. Everyone should know that by now.
What is Safari getting by using Safe Browsing?
The argument would work better if Google wasn't the #1 distributor of scams and malware in the world with adsense. (Which strangely isn't flagged by safe browsing, maybe a coincidence)
Is that actually relevant when only images are user content?
Normally I see the PSL in context of e.g. cookies or user-supplied forms.
> Is that actually relevant when only images are user content?
Yes. For instance in circumstances exactly as described in the thread you are commenting in now and the article it refers to.
Services like google's bad site warning system may use it to indicate that it shouldn't consider a whole domain harmful if it considers a small number of its subdomains to be so, where otherwise they would. It is no guarantee, of course.
Well, using the public suffix list _also_ isolates cookies and treats the subdomains as different sites, which may or may not be desirable.
For example, if users are supposed to log in on the base account in order to access content on the subdomains, then using the public suffix list would be problematic.
Cross domain identity management is a little extra work, but it's far from a difficult problem. I understand the objection to needing to do it when a shared cookie is so easy, but if you want subdomains to be protected from each other because they do not have shared responsibility for each other then it makes sense in terms of privacy & security that they don't automatically share identity tokens and other client-side data.
How does the PSL make any sense? What stops an attacker from offering free static hosting and then making use of their own service?
I appreciate the issue it tries to solve but it doesn't seem like a sane solution to me.
PSL isn't a list of dangerous sites per-se.
Browsers already do various levels of isolation based on domain / subdomains (e.g. cookies). PSL tells them to treat each subdomain as if it were a top level domain because they are operated (leased out to) different individuals / entities. WRT to blocking, it just means that if one subdomain is marked bad, it's less likely to contaminate the rest of the domain since they know it's operated by different people.
I think this only is true if you host independent entities. If you simply construct deep names about yourself with demonstrable chain of authority back, I don't think the PSL wants to know. Otherwise there is no hierarchy the dots are just convenience strings and it's a flat namespace the size of the PSLs length.
Oh - of course this is where I find the answer why there's a giant domain list bloating my web bundles (tough-cookie/tldts).
Does Google use this for Safe Browsing though?
Looks like it? https://developers.google.com/safe-browsing/reference/URLs.a...
This is not about user content, but about their own preview environments! Google decided their preview environments were impersonating... Something? And decided to block the entire domain.
There is no law appointing that organization as a world wide authority on tainted/non tainted sites.
The fact it's used by one or more browsers in that way is a lawsuit waiting to happen.
Because they, the browsers, are pointing a finger to someone else and accusing them of criminal behavior. That is what a normal user understands this warning as.
Turns out they are wrong. And in being wrong they may well have harmed the party they pointed at, in reputation and / or sales.
It's remarkable how short sighted this is, given that the web is so international. Its not a defense to say some third party has a list, and you're not on it so you're dangerous
Incredible
As far as I know there is currently no international alternative authority for this. So definitely not ideal, but better than not having the warnings.
Yes but that's not a legal argument.
You're honor, we hurt the plaintiff because it's better than nothing!
True, and agreed that lawsuits are likely. Disagree that it's short-sighted. The legal system hasn't caught up with internet technology and global platforms. Until it does, I think browsers are right to implement this despite legal issues they might face.
In what country hasn't the legal system caught up?
The point I raise is that the internet is international. There are N legal systems that are going to deal with this. And in 99% of them this isn't going to end well for Google if plaintiff can show there are damages to a reasonable degree.
It's bonkers in terms of risk management.
If you want to make this a workable system you have to make it very clear this isn't necessarily dangerous at all, or criminal. And that a third party list was used, in part, to flag it. And even then you're impeding visitors to a website with warnings without any evidence that there is in fact something wrong.
If this happens to a political party hosting blogs, it's hunting season.
Aw. I saw Jothan Frakes and briefly thought my favorite Starfleet first officer's actor had gotten into writing software later in life.
Be sure to see the team's whole list of Cursed Knowledge. https://immich.app/cursed-knowledge
I love Immich & greatly appreciate the amazing work the team put into maintaining it, but between the OP & this "Cursed Knowledge" page, the apparent team culture of shouting from the rooftops complaints that expose their own ignorance about technology is a little concerning to be honest.
I've now read the entire Cursed Knowledge list & - while I found some of them to be invaluable insights & absolutely love the idea of projects maintaining a public list of this nature to educate - there are quite a few red flags in this particular list.
Before mentioning them: some excellent & valuable, genuinely cursed items: Postgres NOTIFY (albeit adapter-specific), npm scripts, bcrypt string lengths & especially the horrifically cursed Cloudflare fetch: all great knowledge. But...
> Secure contexts are cursed
> GPS sharing on mobile is cursed
These are extremely sane security feature. Do we think keeping users secure is cursed? It honestly seems crazy to me for them to have published these items in the list with a straight face.
> PostgreSQL parameters are cursed
Wherein their definition of "cursed" is that PG doesn't support running SQL queries with more than 65535 separate parameters! It seems to me that any sane engineer would expect the limit to be lower than that. The suggestion that making an SQL query with that many parameters is normal seems problematic.
> JavaScript Date objects are cursed
Javascript is zero-indexed by convention. This one's not a huge red flag but it is pretty funny for a programmer to find this problematic.
> Carriage returns in bash scripts are cursed
Non-default local git settings can break your local git repo. This isn't anything to do with bash & everyone knows git has footguns.
Some of these seem less cursed, and more just security design?
>Some phones will silently strip GPS data from images when apps without location permission try to access them.
That strikes me as the right thing to do?
I think the “cursed” part (from the developers point of view) is that some phones do that, some don’t, and if you don’t have both kinds available during testing, you might miss something?
Huh. Maybe? I don't want that information available to apps to spy on me. But I do want full file contents available to some of them.
And wait. Uh oh. Does this mean my Syncthing-Fork app (which itself would never strike me as needing location services) might have my phone's images' location be stripped before making their way to my backup system?
EDIT: To answer my last question: My images transferred via Syncthing-Fork on a GrapheneOS device to another PC running Fedora Atomic have persisted the GPS data as verified by exiftool. Location permissions have not been granted to Syncthing-Fork.
Happy I didn't lose that data. But it would appear that permission to your photo files may expose your GPS locations regardless of the location permission.
With the Nextcloud app I remember having to enable full file permissions to preserve the GPS data of auto-uploaded photos a couple of years ago. Which I only discovered some months after these security changes went into effect on my phone. That was fun. I think Android 10 or 11 introduced it.
Looking now I can't even find that setting anymore on my current phone. But the photos still does have the GPS data intact.
It's not if it silently alters the file. i do want GPS data for geolocation, so that when i import the images in the right places they are already placed where they should be on the map
> That strikes me as the right thing to do
Yep, and it's there for very goos reasons. However if you don't know about it, it can be quite surprising and challenging to debug.
Also it's annoying when your phones permissions optimiser runs and removes the location permissions from e.g. Google Photos, and you realise a few months later that your photos no longer have their location.
How does it makes sense?
This kind of makes we wish CURSED.md was a standard file in projects. So much hard-earned knowledge could be shared.
You know you can just start doing that in your projects. That's how practice often becomes standard.
> JavaScript date objects are 1 indexed for years and days, but 0 indexed for months.
I don't disagree that months should be 1-indexed, but I would not make that assumption solely based on days/years being 1-indexed, since 0-indexing those would be psychotic.
The only reason I can think of to 0-index months is so you can do monthName[date.getMonth()] instead of monthName[date.getMonth() - 1].
I don't think adding counterintuitive behavior to your data to save a "- 1" here and there is a good idea, but I guess this is just legacy from the ancient times.
That would have a better solution in a date.getCurrentMonth(), in my opinion.
Why so? Months in written form also start with 1, same as days/years, so it would make sense to match all of them.
For example, the first day of the first month of the first year is 1.1.1 AD (at least for Gregorian calendar), so we could just go with 0-indexed 0.0.0 AD.
The Postgres query parameters one is funny. 65k parameters is not enough for you?!
As it says, bulk inserts with large datasets can fail. Inserting a few thousand rows into a table with 30 columns will hit the limit. You might run into this if you were synchronising data between systems or running big batch jobs.
Sqlite used to have a limit of 999 query parameters, which was much easier to hit. It's now a roomy 32k.
Right, for postgres I would use unnest for inserting a non-static amount of rows.
In the past I've used batches of data, inserted into a separate table with all the constraints turned off and using UNNEST, and then inserted into the final table once it was done. We ended up both batching the data and using UNNEST because it was faster but it still let us resume midway through.
We probably should have been partitioning the data instead of inserting it twice, but I never got around to fixing that.
COPY is likely a better option if you have access to the host, or provider-specific extensions like aws_s3 if you have those. I'm sure a data engineer would be able to suggest a better ETL architecture than "shove everything into postgres", too.
Was MERGE too slow/expensive? We tend to MERGE from staging or temporary tables when we sync big data sets. If we were on postgres I think we'd use ... ON CONFLICT, but MERGE does work.
COPY is often a usable alternative.
> PostgreSQL USER is cursed > The USER keyword in PostgreSQL is cursed because you can select from it like a table, which leads to confusion if you have a table name user as well.
is even funnier :D
Saw the long passwords are cursed one. Reminded me of ancient DES unix passwords only reading the first eight characters. What's old is new again...
The one thing I never understood about these warnings is how they don't run afoul of libel laws. They are directly calling you a scammer and "attacker". The same for Microsoft with their unknown executables.
They used to be more generic saying "We don't know if its safe" but now they are quite assertive at stating you are indeed an attacker.
> They are directly calling you a scammer and "attacker".
No they're not. The word "scammer" does not appear. They're saying attackers on the site and they use the word "might".
This includes third-party hackers who have compromised the site.
They never say the owner of the site is the attacker.
I'm quite sure their lawyers have vetted the language very carefully.
"The people living at this address might be pedophiles and sexual predators. Not saying that they are, but if your children are in the vicinity, I strongly suggest you get them back to safety."
I think that might count as libel.
i think it's more akin to "people may have broken in and taken over this house, and within the house there may be sexual predators"
Still asserts that in that house there may be sexual predators. If I lived in that house I wouldnt be happy, and I would want a way of clearing the accusations and proving that there are indeed no sexual predators in my house quicksmart before other people start avoiding it.
You can’t possibly use the “they use the word ‘might’” argument and not mention the death red screen those words are printed over. If you are referring to abidance to the law, you are technically right. If we remove the human factor, you technically are.
This is tricky to get right.
If the false positive rate is consistently 0.0%, that is a surefire sign that the detector is not effective enough to be useful.
If a false positive is libel, then any useful malware detector would occasionally do libel. Since libel carries enormous financial consequences, nobody would make a useful malware detector.
I am skeptical that changing the wording in the warning resolves the fundamental tension here. Suppose we tone it down: "This executable has traits similar to known malware." "This website might be operated by attackers."
Would companies affected by these labels be satisfied by this verbiage? How do we balance this against users' likelihood of ignoring the warning in the face of real malware?
The problem is that it's so one sided. They do what they want with no effort to avoid collateral damage and there's nothing we can do about it.
They could at least send a warning email to the RFC2142 abuse@ or hostmaster@ address with a warning and some instructions on a process for having the mistake reviewed.
> The one thing I never understood about these warnings is how they don't run afoul of libel laws.
I’m not a lawyer, but this hasn’t ever been taken to court, has it? It might qualify as libel.
I know of no such cases, and would love to know if someone finds one.
I worked for a company who had this happen to an internal development domain, not exposed to the public internet. (We were doing security research on our own software, so we had a pentest payload hosted on one of those domains as part of a reproduction case for a vulnerability we were developing a fix for.)
Our lawyers spoke to Google's lawyers privately, and our domains got added to a whitelist at Google.
you only sue somebody poorer than you
It depends, if it's a clear-cut case, then in jurisdictions with a functioning legal system it can be feasible to sue.
Likewise, if it's a fuckup that just needs to be put in front of someone who cares, a lawsuit is actually a surprisingly effective way of doing that. This moves your problem from "annoying customer support interaction that's best dealt with by stonewalling" into "legal says we HAVE to fix this".
Imagine if you bought a plate at Walmart and any time you put food you bought elsewhere on it, it turned red and started playing a warning about how that food will probably kill you because it wasn't Certified Walmart Fresh™
Now imagine it goes one step further, and when you go to eat the food anyway, your Walmart fork retracts into its handle for your safety, of course.
No brand or food supplier would put up with it.
That's what it's like trying to visit or run non-blessed websites and software coming from Google, Microsoft, etc on your own hardware that you "own".
This is the future. Except you don't buy anything, you rent the permission to use it. People from Walmart can brick your carrots remotely even when you don't use this plate, for your safety ofc
I see a lot of comments here about using some browser that will allow ME to see sites I want to see, but I did not see a lot about how do I protect my site or sites of clients from being subjected to this. Is there anything proactive that can be done? A set of checks almost like regression testing? I understand it can be a bit like virus builders using anti virus to test their next virus. But is there a set of best practices that could give you higher probability of not being blocked?
> how do I protect my site or sites of clients from being subjected to this. Is there anything proactive that can be done?
Some steps to prevent this happening to you:
1. Host only code you own & control on your own domain. Unless...
2. If you have a use-case for allowing arbitrary users to publish & host arbitrary code on a domain you own (or subdomains of), then ensure that domain is a separate dedicated one to the ones you use for your own owned code, that can't be confused with your own owned hosted content.
3. If you're allowing arbitrary members of the public to publish arbitrary code for preview/testing purposes on a domain you own - have the same separation in place for that domain as mentioned above.
4. If you have either of the above two use-cases, publish that separated domain on the Mozilla Public Suffix list https://publicsuffix.org/
That would protect your domains from being poisoned by arbitrary publishing, but wouldn't it risk all your users being affected by one user publishing?
Allowing user publishing is an inherent risk - these are good mitigations but nothing will ever be bulletproof.
The main issue is protecting innocent users from themselves - that's a hard one to generalise solutions to & really depends on your publishing workflows.
Beyond that, the last item (Public Suffix list) comes with some decent additional mitigations as an upside - the main one being that Firefox & Chrome both enable more restrictive cookie settings while browsing any domains listed in the public suffix list.
---
All that said - the question asked in the comment at the top of the thread wasn't about protecting users from security risk, but protecting the domain from being flagged by Google. The above steps should at least do that pretty reliably, barring an actual legitimate hack occurring.
Never host your test environments as Subdomains of your actual production domain. You'll also run into email reputation as well as cookie hell. You can get a lot of cookies from the production env if not managed well.
This. I cannot believe the rest of the comments on this are seemingly completely missing the problem here & kneejerk-blaming Google for being an evil corp. This is a real issue & I don't feel like the article from the Immich team acknowledges it. Far too much passing the buck, not enough taking ownership.
Yep. Still I feel bad for them.
I think my comment came across a bit harsh - the Immich team are brilliant. I've hosted it for a long time & couldn't be happier & I think my criticisms of the tone of the article are likely a case of ignorance rather than any kind of laziness or dismissiveness.
It's also in general a thankless job maintaining any open-source project, especially one of this scale, so a certain level of kneejerk cynical dismissiveness around stuff like this is expected & very forgivable.
Just really hope the ignorance / knowledge-gap can be closed off though, & perhaps some corrections to certain statements published eventually.
I think immich.app is the production domain, not cloud?
This may not be a huge issue depending on mitigating controls but are they saying that anyone can submit a PR (containing anything) to Immich, tag the pr with `preview` and have the contents of that PR hosted on https://pr-<num>.preview.internal.immich.cloud?
Doesn't that effectively let anyone host anything there?
No, it doesn't work at all for PRs from forks.
I think only collaborators can add labels on github, so not quite. Does seem a bit hazardous though (you could submit a legit PR, get the label, and then commit whatever you want?).
Exposure also extends not just to the owner of the PR but anyone with write access to the branch from which it was submitted. GitHub pushes are ssh-authenticated and often automated in many workflows.
That was my first thought - have the preview URLs possibly actually been abused through GitHub?
So basically like https://docs.google.com/ ?
Yes, except on Google Docs you can't make the document steal credentials or download malware by simply clicking on the link.
It's more like sites.google.com.
Excellent idea for cost-free phishing.
Story of when it happened to my company: https://news.ycombinator.com/item?id=25802366
Insane that one company can dictate what websites you're allowed to visit. Telling you what apps you can run wasn't far enough.
It's the result of failures across the web, really. Most browsers started using Google's phishing site index because they didn't want to maintain one themselves but wanted the phishing resistance Google Chrome has. Microsoft has SmartScreen, but that's just the same risk model but hosted on Azure.
Google's eternal vagueness is infuriating but in this case the whole setup is a disaster waiting to happen. Google's accidental fuck-up just prevented "someone hacked my server after I clicked on pr-xxxx.imiche.app" because apparently the domain's security was set up to allow for that.
You can turn off safe browsing if you don't want these warnings. Google will only stop you from visiting sites if you keep the "allow Google to stop me from visiting some sites" checkbox enabled.
US congress not functioning for over a decade causes a few problems.
I really don't know how they got nerds to think scummy advertising is cool. If you think about it, the thing they make money on - no user actually wants ads or wants to see them, ever. Somehow Google has some sort of nerd cult that people think its cool to join such an unethical company.
So unethical that they made countless free services that millions of people have relied on every day for years. Do you interface with anyone that's not deep in the software industry? Every regular person I know uses everything Google without any hesitation and no more than a bit of annoyance with ads sometimes. I think they all are pretty happy with the deal and would not switch to a paid ad-free version.
I'm increasingly blown away by takes on here that are so dramatic and militant about things that barely even register to most people.
Google's services, especially their free services, are never really free. It's just that the price tag is so well hidden that ordinary users really believe this. But the HN audience is more technical than that and they see through the smokescreen.
Except for those that are making money off adds directly or indirectly, and who believe in their god given right to my attention and my data.
> I'm increasingly blown away by takes on here that are so dramatic and militant about things that barely even register to most people.
Things 'barely even registering to most people' is not as strong a position as you may think it is. Oxygen barely registers to most people. But take it away and they register it just fine (for a short while). The 'regular' people that you know have been steadily conditioned to an ever worsening experience to the point that they barely recognize the websites they visit when seeing the web with an adblocker for the first time.
On this day, only Google Maps does not have real competitor on Android. Otherwise, it is possible to drop Google and even get better services. Brands are difficult to compete.
Try Mapy. Outperforms Google maps any day.
They created the largest spying instrument in the world that creates hidden profiles (that can never be deleted) documenting web activity, psychological state, medications, etc, etc for billions of people - and have been caught multiple times sharing data with governments (they're probably compromised internally anyway). I would categorize that as unethical. But yeah, you can cheer for the scraps they throw out.
>about things that barely even register to most people.
News flash: This whole website is about things that don't register to most people. It's called hacker news FFS.
In any case, I think a trillion dollar company probably doesn't need defending. They can easily tweak their algorithm to bury this type of stuff; after all this opinion is probably not "relevant" or "useful" to most people.
Turns out it's cool to make lots of money
unfortunately nobody wants to sacrifice anything nowadays so everyone will keep using google, and microsoft, and tiktok and meta and blah blah
It's super simple. Check out all the Fediverse alternatives. How many people that talk a big game actually financially support those services? 2% maybe, on the high end.
Things cost money, and at a large scale, there's either capitalism, or communism.
Absolutely fuck Google
The open internet is done. Monopolies control everything.
We have an iOS app in the store for 3 years and out of the blue apple is demanding we provide new licenses that don’t exist and threaten to kick our app out. Nothing changed in 3 years.
Getting sick of these companies able to have this level of control over everything, you can’t even self host anymore apparently.
> We have an iOS app in the store for 3 years and out of the blue apple is demanding we provide new licenses that don’t exist and threaten to kick our app out.
Crazy! If you can elaborate here, please do.
I write a couple of libraries for creating GOV.UK services and Google has flagged one of them as dangerous. I've appealed the decision several times but it's like screaming into a void.
https://govuk-components.netlify.app/
I use Google Workspace for my company email, so that's the only way for me to get in contact with a human, but they refuse to go off script and won't help me contact the actual department responsible in any way.
It's now on a proper domain, https://govuk-components.x-govuk.org/ - but other than moving, there's still not much anyone can do if they're incorrectly targeted.
Us nerds *really* need to come together in creating a publicly owned browser (non chromium)
Surely among us devs, as we realize app stores increasingly hostile, that the open web is worth fighting for, and that we have the numbers to build solutions?
Uh… we are. Servo and Ladybird. It’s a shit tonne of work.
Firefox should be on that list. It's clearly a lot closer in functionality to Chrome/Chromium than Servo or Ladybird, so it's easier to switch to it. I like that Servo and Ladybird exist and are developing well, but there's no need to pretend that they're the only available alternatives.
And also, it's very feasible to contribute to Firefox. And through it, to Zen Browser, Librewolf, etc. as well.
If you knew how the Mozilla corporation was governed, then you would not think that Firefox should be on the list.
How is it governed?
> It’s a shit tonne of work.
[Sam didn't like that.]
A friend / client of mine used some kind of WordPress type of hosting service with a simple redirect. The host got on the bad sites list.
This also polluted their own domain, even when the redirect was removed, and had the odd side effect that Google would no longer accept email from them. We requested a review and passed it, but the email blacklist appears to be permanent. (I already checked and there are no spam problems with the domain.)
We registered a new domain. Google’s behaviour here incidentally just incentivises bulk registering throwaway domains, which doesn’t make anything any better.
Wow. That scares me. I've been using my own domain that got (wrongly) blacklisted this week for 25 years and can't imagine having email impacted.
I'm fighting this right now on my own domain. Google marked my family Immich instance as dangerous, essentially blocking access from Chrome to all services hosted on the same domain.
I know that I can bypass the warning, but the photo album I sent to my mother-in-law is now effectively inaccessible.
Unless I missed something in the article this seems like a different issue. The article is specifically about the domain "immich.cloud". If you're using your own domain, I'd check to ensure it hasn't been actually compromised by a bonnet or similar in some way you haven't noticed.
It may well be a false positive of Google's heuristics but home server security can be challenging - I would look at ruling out the possibility of it being real first.
It certainly sounds like a separate root issue to this article, even if the end result looks the same.
Just in case you're not sure how to deal with it, you need to request a review via the Google Search Console. You'll need a Google account and you have to verify ownership of the domain via DNS (if you want to appeal the whole domain). After that, you can log into the Google Search Console and you can find "Security Issues" under the "Security & Manual Actions" section.
That area will show you the exact URLs that got you put on the block list. You can request a review from there. They'll send you an email after they review the block.
Hopefully that'll save you from trying to hunt down non-existent malware on a half dozen self-hosted services like I ended up doing.
It's a bit ironic that a user installing immich to escape Google's grip ends up having to create again a Google account to be able to remove their Google account.
No later than last weekend I was comtemplating migrating my family pictures to a self-hosted Immich instance...
I guess a workaround Google's crap would be to put an htpasswd/basic auth in front of Immich, blocking Google to get to the content and flagging it.
Add a custom "welcome message" in Server Settings (https://my.immich.app/admin/system-settings?isOpen=server) to make your login page look different compared to all other default Immich login pages. This is probably the easiest non-intrusive tweak to work around the repeated flagging by Safe Browsing, still no 100% guarantee. I agree that strict access blocking (with extra auth or IP ACL) can work better. Though I've seen in this thread https://news.ycombinator.com/item?id=45676712 and over the Internet that purely internal/private domains get flagged too. Can it be some Chrome + G Safe Browsing integration, e.g. reporting hashes of visited pages?
Btw, folks in the Jellyfin thread tried blocking specifically Google bot / IP ranges (ASNs?) https://github.com/jellyfin/jellyfin-web/issues/4076#issueco... with varying success.
And go through your domain registration/re-review in G Search Console of course.
Out of curiosity, is your Immich instance published as https://immich.example.com ?
Since other browsers, like Firefox, also use the Google Safe Browsing list, they are affected as well.
Looking forward to Louis Rossmann's reaction. Wouldn't be surprised if this leads to a lawsuit over monopolistic behavior - this is clearly abusing their dominant position in the browser space to eliminate competitors in photos sharing.
Who is that and why is his reaction relevant?
He's a right-to-repair activist Youtuber who is quite involved in GrayJay, another app made by this company, which is a video player client for other platforms like YouTube.
I'm not sure why his reaction would be relevant, though. It'll just be another rant about how Google has too much control like he's done in the past. He may be right, but there's nothing new to say.
He wasn't just involved with GrayJay, he's actually a member of FUTO - the company behind Immich and GrayJay. Now read grandparent comment one more time:
> Wouldn't be surprised if this leads to a lawsuit over monopolistic behavior
His reaction also matters because he's basically the public face for the company on YouTube and has a huge following. You've probably seen a bunch of social media accounts with the "clippy" character as their avatar. That's a movement started by Louis Rossman.
A good takeaway is to separate different domains for different purposes.
I had prior been tossing up the pros/cons of this (such as teaching the user to accept millions of arbitrary TLDs as official), but I think this article (and other considerations) have solidified it for me.
For example
www.contoso.com (public)
www.contoso.blog (public with user comments)
contoso.net (internal)
staging.contoso.dev (dev/zero trust endpoints)
raging-lemur-a012afb4.contoso.build (snapshots)
The biggest con of this is that to a user it will seem much more like phishing.
It happened to me a while ago that I suddenly got emails from "githubnext.com". Well, I know Github and I know that it's hosted at "github.com". So, to me, that was quite obviously phishing/spam.
Turns out it was real...
This is such a difficult problem. You should be able to buy a “season pass” for $500/year or something that stops anyone from registering adjacent TLDs.
And new TLDs are coming out every day which means that I could probably go buy microsoft.anime if I wanted it.
This is what trademarks are supposed to do, but it’s reactive and not proactive.
PayPal is a real star when it comes to vague, fake-sounding, official domains.
Real users don't care much about phishing as long as you got redirected from the main domain, though. github.io has been accepted for a long time, and githubusercontent.com is invisible 99% of the time. Plus, if your regular users are not developers and still end up on your dev/staging domains, they're bound to be confused regardless.
Good
Maybe a dumb question but what constitutes user-hosted-content?
Is a notion page, github repo, or google doc that has user submitted content that can be publicly shared also user-hosted?
IMO Google should not be able to use definitive language "Dangerous website" if its automated process is not definitive/accurate. A false flag can erode customer trust.
A website where a user can upload "active code".
The definition of "active code" is broad & sometimes debatable - e.g. do old MySpace websites count - but broadly speaking the best way of thinking about it is in terms of threat model, & the main two there are:
- credential leakage
- phishing
The first is fairly narrow & pertains to uploading server side code or client javascript. If Alice hosts a login page on alice.immich.cloud that contains some session handling bugs in her code, Mallory can add some cute to mallory.immich.cloud to read cookies set on *.immich.cloud to compromise Alice's logins.
The second is much broader as it's mostly about plausible visual impersonation so will also cases where users can only upload CSS or HTML.
Specifically in this case what Immich is doing here is extremely dangerous & this post from them - while I'll give them the benefit of the doubt on being ignorant - is misinformation.
It may be dangerous but it is an established pattern. There are many cases (like Cloudflare Pages) of others doing the same, hosting strangers' sites on subdomains of a dedicated domain (pages.dev for Cloudflare, immich.cloud for Immich).
By preventing newcomers from using this pattern, Google's system is flawed, severely stifling competition.
Of course, this is perfectly fine for Google.
It is but this established pattern is well standardised & documented by the public suffix list project. There's generally two conventions followed for this pattern:
1. Use a separate dedicated domain (Immich didn't do this - they're now switching to one in response to this)
2. List the separate dedicated domain in the public suffix list. As far as I can tell Immich haven't mentioned this.
This is #1 on HN for a while now and I suspect it's because many of us are nervous about it happening to us (or have already had our own homelab domains flagged!).
So is there someone from Google around who can send this along to the right team to ensure whatever heuristic has gone wrong here is fixed for good?
I doubt Google the corporation cares one bit, and any individual employees who do care would likely struggle against the system to cause significant change.
The best we all can do is to stop using Google products and encourage our friends and family to do likewise. Make sure in our own work that we don't force others to rely on Google either.
Given the scale of Google, and the nerdiness required to run Immich, I bet it's just an accident. Nevertheless, I'm very curious as to how senior Google staff looks at Immich, are they actually registering signals that people use immich-go to empty their Google Photos accounts? Do they see this as something potentially dangrous to their business in the long term?
The nerdsphere has been buzzing with Immich for some time now (I started using it a month back and it lives up to its reputation!), and I assume a lot of Googlers are in that sphere (but not neccessarily pro-Google/anti-Immich of course). So I bet they at least know of it. But do they talk about it?
I love Immich but the entire design and interface is so clearly straight up copied from Google photos. It makes me a bit nervous about their exposure, legally.
Can I use this space to comment on how amazing Immich is? I self host lots of stuff, and there’s this one tier above everything else that’s currently, and exclusively, held by Home Assistant and Immich. It is actually _better_ than Google photos (if you keep your db and thumbs on ssd, and run the top model for image search). You give up nothing, and own all your data.
I migrated over from google photos 2 years ago. It has been nothing but amazing. No wonder google has it in its crosshairs.
What model do you recommend for image search?
yeah same, I'm in the process of migrating so I have both google photo and immich, and honestly immich is just as good.
I actually find the semantic search of immich slightly better.
The same thing happened to me earlier this year with a self-hosted instance of Umami Analytics.
https://news.ycombinator.com/item?id=42779544#42783321
Unironically, including a threat of legal action in my appeal on the Google Search Console was what stopped our instance getting flagged in the end.
Could you provide your text? Having same issue for years https://news.ycombinator.com/item?id=45678095
This is crazy, it happened to the SoGO webmailer, standalone or bundled with the mailcow: dockerized stack as well. They implemented a slight workaround where URLs are being encrypted to avoid pattern detection to flag it as "deceiving".
There is no responses from Google about this. I had my instance flagged 3 times on 2 different domains including all subdomains, displaying a nice red banner on a representative business website. Cool stuff!
When the power is concentrated in one hands, those hands will always become the hands of a dictator
This happened to one of our documentation sites. My co-workers all saw it before I did, because Brave (my daily driver) wasn't showing it. I'm not sure if Brave is more relaxed in determining when a site is "dangerous" but I was glad not to be seeing it, because it was a false positive.
This seems related to another hosting site that got caught out by this recently:
https://news.ycombinator.com/item?id=45538760
Not quite the same (other than being an abuse of the same monopoly) since this one is explicitly pointing to first-party content, not user content.
Tangential to the flagging issue, but is there any documentation on how Immich is doing the PR site generation feature? That seems pretty cool, and I'd be curious to learn more.
It's open source, you can find this trivially yourself in less than a minute.
https://github.com/immich-app/devtools/tree/a9257b33b5fb2d30...
Wow. What a rude way to answer.
Sometimes it is also rude to ask without looking the obvious place themselves. It is about signaling that ”my” time is more precious than ”your” time so I let them do that check for me, if I can use someone elses time.
I think we might have hit the inflection point where being rude is more polite. It's not that I want people to be rude to me, it's that I don't want to talk to AI when I intend to be talking to a person, and anyone engaging with me via AI is infinitely more disrespectful than any curse word or rudeness.
These days, when I get a capitalized, grammatically correct sentence — and proper punctuation to boot, there is an unfortunate chance it was written using an AI and I am not engaging fully with a human.
its when my covnersation partner makes human mistakes, like not capitalizing things, or when they tell me i'm a bonehead, that i know i'm talking to a real human not a bot. it makes me feel happier and more respected. i want to interact with humans dammit, and at this point rude people are more likely to be human than polite ones on the internet.
i know you can prompt AIs to make releaistic mistakes too, the arms race truly never ends
Pretty sure Immich is on github, so I assume they have a workflow for it, but in case you're interested in this concept in general, gitlab has first-class support for this which I've been using for years: https://docs.gitlab.com/ci/review_apps/ . Very cool and handy stuff.
Safe Browsing collects a lot of data, such as hashes of URLs (URLs can be easily decoded by comparison) and probably other interactions with web like downloads.
But how effective is it in malware detection?
The benefits seem to me dubious. It looks like a feature offered to collect browsing data, useful to maybe 1% in special situations.
100% agreed. Adblock does this better and doesn’t randomly block image sharing websites
If you block those internal subdomains from search with robots.txt, does Google still whine?
I’ve heard anecdotes of people using an entirely internal domain like “plex.example.com” even if it’s never exposed to the public internet, google might flag it as impersonating plex. Google will sometimes block it based only on name, if they think the name is impersonating another service.
Its unclear exactly what conditions cause a site to get blocked by safe browsing. My nextcloud.something.tld domain has never been flagged, but I’ve seen support threads of other people having issues and the domain name is the best guess.
I'm almost positive GMail scanning messages is one cause. My domain got put on the list for a URL that would have been unknowable to anyone but GMail and my sister who I invited to a shared Immich album. It was a URL like this that got emailed directly to 1 person:
https://photos.example.com/albums/xxxxxxxx-xxxx-xxxx-xxxx-xx...
Then suddenly the domain is banned even though there was never a way to discover that URL besides GMail scanning messages. In my case, the server is public so my siblings can access it, but there's nothing stopping Google from banning domains for internal sites that show up in emails they wrongly classify as phishing.
Think of how Google and Microsoft destroyed self hosted email with their spam filters. Now imagine that happening to all self hosted services via abuse of the safe browsing block lists.
if it was just the domain, remember that there is a Cert Transparency log for all TLS certs issued nowadays by valid CAs, which is probably what Google is also using to discover new active domains
It doesn’t seem like email scanning is necessary to explain this. It appears that simply having a “bad” subdomain can trigger this. Obviously this heuristic isn’t working well, but you can see the naive logic of it: anything with the subdomain “apple” might be trying to impersonate Apple, so let’s flag it. This has happened to me on internal domains on my home network that I've exposed to no one. This also has been reported at the jellyfin project: https://github.com/jellyfin/jellyfin-web/issues/4076
In my case though, the Google Search Console explicitly listed the exact URL for a newly created shared folder as the cause.
https://photos.example.com/albums/xxxxxxxx-xxxx-xxxx-xxxx-xx...
That's not going to be gleaned from a CT log or guessed randomly. The URL was only transmitted once to one person via e-mail. The sending was done via MXRoute and the recipient was using GMail (legacy Workspace).
The only possible way for Google to have gotten that URL to start the process would have been by scanning the recipient's e-mail.
Not quite. Presumably the recipient clicked the link, at which point their browser knows it and, depending on browser and settings, may submit it to Google to check if it's "safe": https://support.google.com/chrome/answer/9890866#zippy=%2Cen...
Well, that's potentially horrifying. I would love for someone to attempt this in as controlled of a manner as possible. I would assume it's possible for anyone using Google DNS servers to also trigger some type of metadata inspection resulting in this type of situation as well.
Also - when you say banned, you're speaking of the "red screen of death" right? Not a broader ban from the domain using Google Workplace services, yeah?
> Also - when you say banned, you're speaking of the "red screen of death" right?
Yes.
> I would love for someone to attempt this in as controlled of a manner as possible.
I'm pretty confident they scanned a URL in GMail to trigger the blocking of my domain. If they've done something as stupid as tying GMail phishing detection heuristics into the safe browsing block list, you might be able to generate a bunch of phishy looking emails with direct links to someone's login page to trigger the "red screen of death".
This reminds me of another post where a scammer sent a gmail message containing https://site.google.com/xxx link to trick users into click, but gmail didn't detect the risk.
Chrome sends visited urls to Google (ymmv depending on settings and consents you have given)
Yes, my family Immich instance is blocked from indexing both via headers and robots.txt, yet it's still flagged by Google as dangerous.
I'm kind of curious, do you have your own domain for immich or is this part of a malware-flagged subdomain issue? It's kind of wild to me that Google would flag all instances of a particular piece of self-hosted software as malicious.
G would flag _some_ instances.
Possible scenario:
- A self-hosted project has a demo instance with a default login page (demo.immich.app, demo.jellyfin.org, demo1.nextcloud.com) that is classified as "primary" by google's algorithms
- Any self-hosted instance with the same login page (branding, title, logo, meta html) becomes a candidate for deceptive/phishing by their algorithm. And immich.cloud has a lot of preview envs falling in that category.
BUT in Immich case its _demo_ login page has its own big banner, so it is already quite different from others. Maybe there's no "original" at all. The algorithm/AI just got lost among thousands of identically looking login pages and now considers every other instance as deceptive...
Google often marks my homelab domains as dangerous which all point to an A record that is in the private IP space, completely inaccessible to the internet.
Makes precisely zero sense.
Them maintaining a page of gotchas is a really cool idea - https://immich.app/cursed-knowledge
> There is a user in the JavaScript community who goes around adding "backwards compatibility" to projects. They do this by adding 50 extra package dependencies to your project, which are maintained by them.
This is a spicy one, would love to know more.
It links to a commit; the removed deps are by GitHub user ljharb.
Ran a clickbait site, and got flagged for using a bunch of 302 redirects instead of 301s. Went from almost 500k uniques a month to 1k.
During the appeal it was reviewed from India, and I had been using geoblocking. This caused my appeal to be denied.
I ended up deploying to a new domain and starting over.
Never caught back up.
Congrats on this great choice of business endeavor
Regarding how Google safe browsing actually works under the hood, here is a good writeup from Chromium team:
https://blog.chromium.org/2021/07/m92-faster-and-more-effici...
Not sure if this is exactly the scenario from the discussed article but it's interesting to understand it nonetheless.
TL;DR the browser regularly downloads a dump of color profile fingerprints of known bad websites. Then when you load whatever website, it calculates the color profile fingerprint of it as well, and looks for matches.
(This could be outdated and there are probably many other signals.)
I can't imagine that lasted more than 30 seconds after they made a public blog post about how they were doing it.
I am confused if the term "self-hosted" means the same thing to them as it means to me, not sure if I'm following.
I tried to submit this, but the direct link here is probably better than the Reddit thread I linked to:
https://old.reddit.com/r/immich/comments/1oby8fq/immich_is_a...
I had my personal domain I use for self-hosting flagged. I've had the domain for 25 years and it's never had a hint of spam, phishing, or even unintentional issues like compromised sites / services.
It's impossible to know what Google's black box is doing, but, in my case, I suspect my flagging was the result of failing to use a large email provider. I use MXRoute for locally hosted services and network devices because they do a better job of giving me simple, hard limits for sending accounts. That way if anything I have ever gets compromised, the damage in terms of spam will be limited to (ex) 10 messages every 24h.
I invited my sister to a shared Immich album a couple days ago, so I'm guessing that GMail scanned the email notifying her, used the contents + some kind of not-google-or-microsoft sender penalty, and flagged the message as potential spam or phishing. From there, I'd assume the linked domain gets pushed into another system that eventually decides they should blacklist the whole domain.
The thing that really pisses me off is that I just received an email in reply to my request for review and the whole thing is a gas-lighting extravaganza. Google systems indicate your domain no longer contains harmful links or downloads. Keep yourself safe in the future by blah blah blah blah.
Umm. No! It's actually Google's crappy, non-deterministic, careless detection that's flagging my legitimate resources as malicious. Then I have to spend my time running it down and double checking everything before submitting a request to have the false positive mistake on Google's end fixed.
Convince me that Google won't abuse this to make self hosting unbearable.
> I suspect my flagging was the result of failing to use a large email provider.
This seems like the flagging was a result of the same login page detection that the Immich blog post is referencing? What makes you think it's tied to self-hosted email?
I'm not using self hosted email. My theory is that Google treats smaller mail providers as less trustworthy and that increases the odds of having messages flagged for phishing.
In my case, the Google Search Console explicitly listed the exact URL for a newly created shared album as the cause.
https://photos.example.com/albums/xxxxxxxx-xxxx-xxxx-xxxx-xx...
I wish I would have taken a screenshot. That URL is not going to be guessed randomly and the URL was only transmitted once to one person via e-mail. The sending was done via MXRoute and the recipient was using GMail (legacy Workspace).
The only possible way for Google to have gotten that URL to start the process would have been by scanning the recipient's e-mail. What I was trying to say is that the only way it makes sense to me is if Google via GMail categorized that email as phishing and that kicked off the process to add my domain to the block list.
So, if email categorization / filtering is being used as a heuristic for discovering URLs for the block list, it's possible Google's discriminating against domains that use smaller email hosts that Google doesn't trust as much as themselves, Microsoft, etc..
All around it sucks and Google shouldn't be allowed to use non-deterministic guesswork to put domains on a block list that has a significant negative impact. If they want to operate a clown show like that, they should at least be liable for the outcomes IMO.
I'm in a similar boat. Google's false flag is causing issues for my family members who use Chrome, even for internal services that aren't publicly exposed, just because they're on related subdomains.
It's scary how much control Google has over which content people can access on the web - or even on their local network!
It's a good opportunity to recommend Firefox when you can show a clear abuse of position
Firefox uses the same list.
Wonder if there would be any way to redress this in small claims court.
There's a reason GitHub use github.io for user content.
They're using a different TLD (.cloud / .app). But IIRC, GH changed to avoid cookies leaking with user created JS running at their main domain.
I've rarely seen a HN comment section this overwhelmingly wrong on a technical topic. This community is usually better than this.
Google is an evil company I want the web to be free of, I resent that even Firefox & Safari use this safe browsing service. Immich is a phenomenal piece of software - I've hosted it myself & sung its praises on HN in the past.
Put putting aside David vs Goliath biases here, Google is 100% correct here & what Immich are doing is extremely dangerous. The fact they don't acknowledge that in the blog post shows a security knowledge gap that I'm really hoping is closed over the course of remediating this.
I don't think the Immich team mean any harm but as it currently stands the OP constitutes misinformation.
> what Immich are doing is extremely dangerous
I've read the article and don't see anything dangerous, much less extremely so. Care to explain?
They're auto-deploying PRs to a subdomain of a domain that they also use for production traffic. This allows any member of the public with a GitHub account to deploy any arbitrary code to that subdomain without any review or approval from the Immich team. That's bad for two reasons:
1. PR deploys on public repos are inherently tricky as code gains access to the server environment, so you need to be diligent about segregating secrets for pr deployments from production secret management. That diligence is a complex & continuous undertaking, especially for an open source project.
2. Anyone with a GitHub account can use your domain for phishing scams or impersonation.
The second issue is why they're flagged by Google (he first issue may be higher risk to the Immich project but it's out of scope for Google's safe browsing service).
To be clear: this isn't about people running their own immich instance. This is about members of the public having the ability to deploy arbitrary code without review.
---
The article from the Immich team does mention they're switching to using a non-production domain (immich.build) for their PR builds which does indicate to me they somewhat understand the issue (though they've explained it badly in the article), but they don't seem to understand the significance or scope.
> This allows any member of the public with a GitHub account to deploy any arbitrary code to that subdomain without any review or approval from the Immich team.
This part is not correct: the "preview" label can be set only by collaborators.
> a subdomain of a domain that they also use for production traffic
To clarify this part: the only production traffic that immich.cloud serves are static map tiles (tiles.immich.cloud)
Overall, I share your concerns, and as you already mentioned, a dedicated "immich.build" domain is the way to go.
> This part is not correct: the "preview" label can be set only by collaborators.
That's good & is a decent starting point. A decent second step might be to have the Github Actions workflow also check the approval status of the PR before deploying (requiring all collaborators to be constantly aware that the risk of applying a label is similar to that of an approval seems less viable)
I don't want Google to abuse the world wide web. It is time for real change - a world without Google. A world with less Evil.
Curious if anyone had an instance where this blocking mechanism saved them. I can’t remember a single instance in last 10 years
I've had it work for me several times. Most of the time following links/redirects from search engines, ironically a few times from Google itself. Not that I was going to enter anything (the phishing attempts themselves were quite amateurish) but they do help in some rare cases.
When I worked customer service, these phishing blocks worked wonders preventing people from logging in to your-secure-webmail.jobz. People would be filling in phishing forms days after sending out warnings on all official channels. Once Google's algorithm kicked in, the attackers finally needed to switch domains and re-do their phishing attempts.
Your parents probably have
google: we make going to the DMV look delightful by comparison!
They are not the government and should not have this vast, unaccountable monopoly power with no accountability and no customer service.
the government probably shouldn't either?
At least the government is normally elected.
Most of it kind of isn't. When was the last election for FCC commissioners or US Attorney General or federal district court judges?
Honestly, where do people live that the DMV (or equivalent - in some states it is split or otherwise named) is a pain? Every time I've ever been it has been "show up, take a number, wait 5 minutes, get served" - and that's assuming website self-service doesn't suffice.
I have no idea what immich is or what this post says, but I LOVE that this company has a collection of posts called, “Cursed Knowledge.”
I’m launching a web version for an online game. What to do to prevent this from happening?
Install your non-self generated SSL certificate correctly, and make sure users can't upload arbitrary content to your domain.
Is there any linkage to the semifactoid that immich Web gui looks very like Google Photos or is that just one of the coincidences?
Not a coincidence, Immich was started as a personal replacement for Google Photos.
The coincidence here would be google flagging it as malware, not the origin story of the look and feel.
Oh my bad, I severely misinterpreted your comment.
They have to fix their SSL certs. "Kubernetes Ingress Controller Fake Certificate" aint gonna cut it.
I’m also self hosting gitea and pertainer and I’m trying this issue every few weeks. I appeal, they remove the warning, after a week is back. This is ongoing for at least 4 years. I have more than 20 appeals all successfully removing the warning. Ridiculous. I heard legal action is the best option now, any other ideas?
"might trick you into installing unsafe software"
Something Google actively facilities with the ads they serve.
First thing I do when I start to use a browser for the first time is making sure 'Google Safe Browsing' feature is disabled. I don't need yet another annoyance while I browse the web, especially when it's from Google.
SOB
I think the other very interesting thing in the reddit thread[0] for this is that if you do well-known-domain.yourdomain.tld then you're likely to get whacked by this too. It makes sense I guess. Lots of people are probably clicking gmail.shady.info and getting phished.
0: https://old.reddit.com/r/immich/comments/1oby8fq/immich_is_a...
So we can't use photos or immich or images or pics as a sub-domain, but anything nondescript will be considered obfuscated and malicious. Awesome!
> The most alarming thing was realizing that a single flagged subdomain would apparently invalidate the entire domain.
Correct. It works this way because in general the domain has the rights over routing all the subdomains. Which means if you were a spammer, and doing something untoward on a subdomain only invalidated the subdomain, it would be the easiest game in the world to play.
malware1.malicious.com
malware2.malicious.com
... Etc.
And yet if you start typing 192 in chrome, first suggested url is 192.168.l00.1
I’d say this is a clear slight from Google, using their Chrome browser because something or someone is inconveniencing another part of their business, google cloud / google photos.
They did a similar thing with the uBlock Origin extension, flagging it with “this extension might be slowing down your browser” in a big red banner in the last few months of manifest v2 on Chrome. After already having to upload the extension yourself to Chrome cause they took it off the extension store cause it was inhibiting on their ad business.
Google is a massive monopolistic company who will pull strings on one side of their business to help another.
With only Firefox not being based on Chromium and still having manifest v2 the future (5 to 10 years from now) looks bleak. With only 1 browser like this web devs can phase it out slowly by not taking it into consideration when coding or Firefox could enshittify to such an extent because of their manifest v2 monopoly that even that wont make it worth it anymore.
Oh and for the ones not in the know, Manifest is the name of a javascript file manifest.js that decides what browser extensions can and cant modify and the “upgrade” from manifest v2 to v3 has made it near impossible for adblockers to block ads.
Immich is run by FUTO, an organization affiliated with the American far right.
Fascist-run projects present grave security concerns and should be treated as such; flagging their site as dangerous is a reasonable response.
I heard the CEO has a Hitler bedspread and Mussolini tattoo on the far-right of his right buttock.