It is ironic that the very link [1] you provided proves you wrong. The top 5 countries of origin doing IP scanning in the last seven days are China (120k), India (67k), US (52), Iran (44k), and Russia (27k).
I see your point. But then how is the accusation 'west scans internet' connected to 'see this map of countries of origin'? Because I thought he would back up this claim with this source/second paragraph.
If other people (and arguably other govt's) are scanning too, then saying 'west scans internet' seems somewhat superficial. Not that I deny western state actors scanning the internet, its just that everybody does it.
Meh, the context was a western gov scanning the internet.
We tend to hold them to a higher standard than the ones who much more shamelessly operate pseudo-blackhat hacking teams. The west at least tries to maintain a sheen of legality. Or morality. Or whatever.
Do you think the Chinese government tells their citizens that they are shamelessly "operating pseudo-blackhat hacking teams"? No, of course not, just like yout Government doesn't tell you that either. The only reason you think the West is the only trying to maintain "a sheen of legality" is that their voice is the only one you're listening to.
Uhh I don't live in a country with a great firewall. The intel agencies get criticized very heavily in my information bubble. The shady stuff the NSA does is pretty well known. It's in movies all the time too, so I'm sure the mainstream even understands that.
And yet despite all the messed up stuff they do every day they still get held to a higher standard.
The accusation was never made that the west scans the internet. "organizations that scan the entire internet and feed the data to western governments." As an analogy, the west buys iPhones, but it doesn't necessarily manufacture them.
Right, also the source IP of a port scan doesn't say anything about who has initiated that scan. If I were a state actor, I'd do my port scanning from machines in a different jurisdiction for sure.
No, you can't. They have a long, well-established history of concealing their undercover agents. The fact that this is not perfect doesn't mean that they don't make the effort, or that you're doing anything other than fooling yourself if you think that all traffic by a national intelligence agency comes from the netblocks assigned to those countries.
Yeah, could be. But still that does not support the argument of "feed the data to western governments".
When you say: "Look, the people from village A north are stealing apples from the city orchard. Here is a list of apple thieves and the direction (N,E,S,W) from which they came." And this list shows that it appears to be majorly the directions E,S,W (so not directly from village A). Then how is this an argument?
It just shows that everybody steals apples, making the accusation "villagers of A are to blame" superficial. That's the point it tried to make.
> But there are also other reasons your conclusion is wrong I think
I would be interested on why my conclusion is wrong. At best, one could draw nothing from the data as it does not show any relation to state actors. And if this conclusion is drawn, then why did `mike_d` blame the western state actors in the first place?
Think for a second about this: Did you think that the link `mike_d` provided supported the argument "... feed the data to western governments" with the emphasis on 'western'?
Funny enough I did a similar thing for my country (Austria). Found quite a few strange things and even made a collage of screenshots of all webservers hosted in Austria - https://blog.haschek.at/2019/i-scanned-austria.html
To be clear, they said "web servers" not "websites". They just pulled a list of all public IP blocks registered to the country and opened port 80/443 on each IP address and took a screenshot. It's by no means a list of the websites hosted on those servers.
You could get somewhat closer by inspecting public DNS records for those IP addresses and then attempting to load each site by DNS name, but it still wouldn't be a complete index of all websites in the country. I'm thinking that's impossible to collect, or at least very nearly.
First - that vulnerabilities are 'known' does not mean any specific instance of vulnerability is 'known'.
Second - that 'most attacks occur some other way' isn't hugely relevant. We don't 'not check the door locks' because most criminals 'go in through the window'.
Having a government entity knock on the doors and remind folks that they have a problem gives the issue impetus, and even legitimacy within the organization aka instead of 'powerless IT figure from sector 8G' saying we have a problem, now, it's the Government saying 'you should to fix this' thereby giving execs the mandate to spend on it.
This is exactly what the government should be doing - it's proportional, non-invasive, note hugely expensive or complicated, they're not making legal requirements here (because none are needed) etc..
Your litany of solutions is not comprehensive, moreover, item #5 'penalize those for not appropriately respond to bug bounty' is a bit glib - this would definitely be government overstepping their bounds. There are always bugs in software. Weighing the risk v. consequences is not something gov can do.
GDPR, SEC, HIPAA, NYDFS and NYSE all mandate risk management measures if not outright penalize companies and citizens for data breaches after the fact which unfortunately means your Grandmas Syphillis medication has to hit Twitter before there’s intervention.
Without strong financial penalties or an impetus to fix at least critical vulns earlier we’ll continue with the status quo.
I don't want that for you, your Grandma or my own. You shouldn’t want it either.
> 1. Over 80% of breaches happen because of KNOWN but unfixed vulnerabilities.
EQUIFAAAAAAAAAAAAAX!!!!!!!!!
(Yes, the Equifax hack was due to a widely-known vulnerability in Apache Commons that apparently the DHS warned about but Equifax didn't bother to patch it.)
Also, knowing that hospital equipment still runs Windows XP (with some sturdy-but-aged machines running Windows 2000), I'm not sure if there's any good benefit for this. Sure, small businesses might take action on a genuine oversight but larger businesses tend to know already that their systems are insecure (even when taking state-level/sponsored attack out of the equation).
> 1. Over 80% of breaches happen because of KNOWN but unfixed vulnerabilities.
This reason only makes sense to me if I assume that all KNOWN vulnerabilities are (and remain) UNFIXED. Assuming otherwise doesn't make sense because I can't tell how many attacks the KNOWN and FIXED vulnerabilities prevented.
> Most attacks lead with phishing and account takeovers not software vulns.
This might be true, but you seem to suggest that we can only concentrate on preventing one type of attack at a time, and therefore we should only pick defensive strategies for the most common attack,
To clarify, I’m saying governments and regulatory bodies should improve mandates for fixing critical issues with the highest risk first (ie. Remediation).
It’s the same reason state govs in the US mandate car insurance or bonds for drivers.
Companies like people have limited resources, time and money so they should focus on where the risk lies.
I know Germany provides the same service as well, but I don't know how fleshed out it is really. So far all the mails they sent me have been not very helpful.
The intention is good, but in practice I think it's mostly useless because:
* The reports go to the AS operator, who in most cases are not the actual admins of the vulnerable software. Some hosting providers such as Hetzner and Manitu have scripts in place to forward reports to the respective customers, but most don't since it involves a lot of parsing of the email (which is not in an easily machine readable format).
* The emails often warn about security issues that may not actually be problematic (i.e. merely warning about some open port that may be intentionally open, and especially if you operate, say, a honeypot), with no way to opt out for specific hosts/ports. So you can only really filter them entirely in your mail client which I think most people do.
Not as annoying as getting DDoS'ed with amplification attacks because some people can't properly configure their servers... (Also I doubt the BND does this, as another commenter pointed out.)
That depends on whether the BND are testing that it could be used in an attack, or just seeing a port is open. Having UDP/11211 could mean you're running a vulnerable memcached service, but not necessarily so.
Hah - one of my first ever network programming tasks was to do this at a UK hosting company. That and SMTP relays. Good that (some) governments are wise enough to try to keep this sort of thing in check.
I hope they aren't using a perl script triggered by a cronjob on a hand-rolled VM though..
I have some doubts. For example, if they are just outputting the scan results from some tool with a high false positive rate, how is that helpful? It's a waste of time and money for the government. Bug bounty programs have the same issue that probably most bugs found are trash results from a scanning tool.
On the other hand, a custom built tool that tries to find the most serious known vulnerabilities with a low false positive rate would probably be a good thing for the government to run.
"Who can receive services? Federal, state, local, tribal and territorial governments, as well as public and private sector critical infrastructure organizations."
However, methinks US definition of critical infrastructure organizations, both public and private, will be quite broad.
Word on the grapevine is saying that Google is doing similar. One of the "perk" of being a well-known DNS resolver (8.8.8.8) is getting an early notification whenever a server goes "online" on the internet.
Someone types in your new server/domain, like "ijustmadethissite.com", or "newlocation.existingsite.com"
For their computer to resolve this domain name, it's going to call out to a DNS server, of which Google hosts a major one. It can be assumed that they log these names, and can then use that as a "notification" for a site coming up.
But what does that have to do with scanning webservers for vulnerabilities, do they do something with the "newly seen sites", and if so is it documented what they do for scanning?
Because if the vulnerability involves an HTTP request, then the Host header needs to have the domain name of the target website.
So you need: IP address and port for the TCP headers, and the domain name to go in the TCP packet content.
One example of a vulnerability would be having phpMyAdmin with a database password hardcoded and no login needed. Without the domain name it would still be impossible to access. (Of course, domain names shouldn't be considered secret so this would be a very insecure setup.)
I think it's illegal under the Computer Fraud and Abuse act. Also, what should the government do when it finds something? What if the site operators are unresponsive or cannot be contacted? There are a lot of practical problems.
> I think it's illegal under the Computer Fraud and Abuse act.
Things that are illegal for individuals to do aren't necessarily illegal for governments to do. This is a reason why the government should be vigorously doing this, rather than leaving it to private citizens, who risk being charged under the Computer Fraud and Abuse Act.
-----
> Also, what should the government do when it finds something?
It should contact the site operator.
-----
> What if the site operators are unresponsive or cannot be contacted?
I would imagine that in the case that site operators couldn't be contacted, they wouldn't be contacted.
> What if the site operators are unresponsive or cannot be contacted?
This seems like only a minor problem. If people are unresponsive, then oh well, they tried to tell you you're hacked. If the site owner cannot be determined, they can email your ISP. This seems to work well for "one of your customers is torrenting movies", and since every ISP is known by definition (thanks, IP addresses), it should be fairly straightforward to get that message to the actual customer. (Send it with the invoice; if the customer doesn't pay invoices, then it's easy to resolve the hacked site. You were shutting them off anyway.)
Everything's illegal under the CFAA. It's an old bad overreaching law that should be repealed. The government rarely prosecutes itself though, so that's no reason why. Unfortunately, the culture in the US is such that the populace would freak out if the government tried to do such a thing, never mind practical surmountable issues.
The way I read the article, they're actually collecting vulnerability information. So they check a site with Version X running on it, and detect the vuln; then they later see Version Y, without the vuln, and update their vulnerability database.
Nothing in the article suggests that they contact site-owners (I haven't re-read the article, so might be wrong).
I'm not sure why you think it's a potential violation of CFAA to connect to a public server and probe it. There's no suggestion of unauthorized access; that would involve exploiting vulnerabilities they find, and that would be unauthorized access.
Something tells me that even with the somewhat stretched version of extraterritoriality that the US claims about laws like CFAA, they wouldn't try applying that to their closest intelligence/defence partner country operating largely domestically...
We have detected a dangerous virus or service in your hosting environment. Conspiracy theorists and foreign state actors often use these types of methods to spread fake news and influence our elections. These are serious threats to our Democracy, but Fatherland Security is here to help you through this difficult time. Your local neighborhood Security Helper will be at your home in the next few minutes to assist you in removing the dangerous HTTP service. For your safety, please stay away from all doors and windows.
Way back in the early 2000s the FBI contacted a company I was working for to inform us that someone was hosting Disney movies on our servers. So something like this is at least sort of happening.
RFC 2142 role mailboxes are critical. It's pretty telling that the webhosts that ignore the RFC are the ones I constantly find problems at. Godaddy not only routinely hosts all kinds of evil, but they make reporters jump through hoops to let them know they need to clean up their mess. If you do try to email abuse@godaddy.com they'll ignore your report and kick back an autoresponder message telling you to fill out a shitty web form instead. Nothing like making more work for people who already went out of their way to let you know you're causing problems for everyone else.
I'm reading that the UK government is spying on us, and their retrospective plausible excuse is that they are scanning web servers for, erm, vulnerabilities.
No, I don't think that the government is here to help. It allows itself only to maintain force, that it then uses to forcibly extract wealth from its herd, er, sorry citizens.
This is a fair point - in organisational terms it'd be better if NCSC was under a non-ministerial body, independent of political influence and control. Similar format to a university, maybe.
It tells you that the crowd don't want to read unsubstantiated cynicaler-than-thou hot takes on HN.
Downvoting "It's raining because Soros and his globalist Jewish cabal control the weather" does not mean I disagree that it's raining but the edit always comes in [downvoters can't handle the TRUTH, stay classy HN] or similar.
e.g. how is scanning for vulnerabilities "spying on us"? How is scanning for vulerabilities "forcibly extracting wealth"? How is informing people of vulernabilities "not here to help"? It's a thinly disguised flamewar comment, not a comment on the topic.
>> e.g. how is scanning for vulnerabilities "spying on us"?
To play Devil's advocate: once you discover a vulnerability you always have two options: report it and have it fixed, or exploit it for your own gain. You charitably assume that government is somehow obligated to chose the former, while in reality in some cases it might choose the latter.
> To play Devil's advocate: once you discover a vulnerability you always have two options: report it and have it fixed, or exploit it for your own gain. You charitably assume that government is somehow obligated to chose the former, while in reality in some cases it might choose the latter.
This assumes that the government that wants to compromise a domestic host can't do it in a way that is a lot more deniable than porting scanning you from a gov owned IP range.
If the government wants to find and exploit a vulnerability they likely will find a way they don't need some loose cover story for it.
I'd say they aren't doing it wrong 100% of the time. They still massively cock up from time to time, e.g. their anti-encryption campaigns, the stupid attempt to require ID for porn, the disastrous NHS digitisation.
But the gov.uk website is pretty good and they did replace IT with computing in schools.
I believe Alex Van Someran recently took over as head of the UK NCSC. He's someone that I trust to make the right decisions, so I'm quite glad of this fact.
(NOTE: I have no idea if this specific link is related to Alex or anything he's done)
Agreed, but if the US Government were doing this there would be outcry of "spying" and "Government overreach". And before anyone says that the US Gov has lost its trust, let me remind you that UK has GCHQ.
NCSC is the public "arm" of GCHQ, they provide cyber-security guidance to businesses and the general public etc. They are a great source of information for current best-practice regarding cyber security.
Sure, if you value authoritarianism and an intrusive nanny state. The government jiggling the door handles of everyone's house to see if it's unlocked crosses a huge line.
"nanny state" is a purposefully skewed statement that pre-presumes that doing something for the common good is always bad. It's a lazy way of not making an argument.
Why is scanning web servers for vulnerabilities bad?
Yeah, scanning for vulnerabilities in a controlled way isn't bad
I suspect those opposing it are the ones that eventually get caught with glaring vulnerabilities and then we have to hear BS like "they care for security and privacy" when they didn't even use password hashes
scanning for and reporting vulnerable web servers does nothing to limit someone's personal choice to operate one. I just hope they make the data public so that I can make the personal choice to block traffic to/from people who make the personal choice to operate insecure devices on the global internet.
What's wrong with asking first and letting the web operator opt in?
The gist of your argument is if I go up and try to pick your pocket but say my intentions are only to help you from real pickpockets, there's nothing but your personal choice to walk on public sidewalk and should just accept it.
the people who would opt in aren't likely to be the problem. The problem with your pickpocket example is that you lose something when someone picks your pocket, but you lose nothing when someone checks to see what ports are open.
In fact, that's something that's already happening all the time anyway. The only difference is that in this case the person checking for your failures to secure your devices will notify you of the problem instead of exploiting your devices like everyone else will (assuming that they haven't already).
This should not only help people secure their devices, but it should also make the internet a better place for everybody.
Who gets to pay for all the extra traffic they send? the time spent by security guys to review the false positive attack logs they generate? the time spent by operators to bring the services back online when the government probing crashes something?
I get it, you don't like the idea of taxes, but fortunately most people are glad for them and the services they provide.
If this service causes a bunch of crashes (somehow) or they end up DoSing someone they should be responsible for the harm that they cause, but since these scans are no different that what criminals are already doing every day I don't imagine it'll be a huge problem unless they really screw something up.
I'd also guess that the costs in both time and money spent on the traffic generated by DDoS attacks, malware infections, and phishing sites are much much greater than the costs for 'security guys' to review logs, safely automate scans, and notify webhosts of problems. This is a sensible measure that should save massive amounts of time and money for people all around the globe and make the internet better for UK citizens in the process.
>Why is scanning web servers for vulnerabilities bad? //
Not the OP.
I think it's fine in general with one big proviso, that they change the law first to make it lawful.
With a different government it would look more benevolent, with the current government growing ever-more fascist--having now found a surreptitious way to ditch the ECHR, for example--it gets somewhat worrying.
Why is asking for permission first bad? The CISA does this very thing, but businesses have to explicitly ask first and consent unlike the UK. That's the difference between a nanny state policy and one that respects choice and the property rights of others.
You clearly have a very skewed idea of what socialism is. Would you consider parks or public schools socialist too as they also contribute to the common good of society.
Some weaknesses of the computer system intrusion/house intrusion analogy:
* It is pretty obvious to the user if their door is locked, so they don't need pentesters to help them figure it out.
* Houses aren't under attack from the entire planet at all times.
* It not that uncommon to have circumstances arranged such that if someone does barge into your house, you know about it.
If the local government wanted to do something that is closer to to what's going on here -- maybe go door to door offering a security assessment for non-obvious stuff -- that might be a well-received service.
Our local government gave me a call because a neighbor asked for a wellness check because we hadn’t plowed our driveway when we were in California. So it does actually happen kind of.
Jiggling door handles without consent is a defacto criminal act. It's no different if I tried to pick your wallet as you walked down the street and said, "better me than a criminal..." then flashed my badge.
CISA will jiggle your door handles for free, if you ask and consent first. Web server operators who aren't asking for vuln assessments aren't apt to keep them regularly patched to begin with.
> Jiggling door handles without consent is a defacto criminal act.
Connecting to a webserver using HTTP is not a criminal act, under any colour of the law. If you have a listening port open to the internet, you are inviting connections.
Picking pockets is stealing; this is more like saying "Hello!" to someone who is standing in their own open doorway, and observing their response.
I don't think there's anything in the article about this programme providing server operators with reports. They're not trying to save operators from themselves.
Opt-in is generally more fair than opt-out, but in this instance it makes sense - they are not checking personal property, they are checking publicly facing webservers. They are not doing it for the server owners benefit, they are doing it to help keep people secure. Servers that aren't being patched properly are exactly the servers that are a security issue waiting to happen, that such a security force should be identifying and telling to buck their ideas up.
I suppose the differences in how those two equivalent departments approach this, likely come from national mindset differences, and the political differences they cause. At least it seems reasonable to me: that in Washington people might all agree that the right to decide if you are tested is more important than finding insecure webservers, whilst in London people might well all agree on the opposite.
It's more like the government driving around neighbourhoods and doing a survey if you have solar roofs or not, used for a reasonable common good purpose, while letting you know they're doing it and letting you opt put.
I once ranted loudly that governments should be doing this for free. That governments should be assembling the best team of pentesters to pentest everything they can possibly find within their jurisdiction.
I've also ranted about this, and how it should be one of the NSA's top priorities (including doing it for our allies).
It's interesting because there are two main methods for what to do when you find a vulnerability: 1) hold onto it so you can later use it as a weapon or 2) disclose it and patch it. The offensive method has problems because as soon as you use it you are disclosing it. It also has the issue that your enemies may be able to (are likely to) find the same vulnerability and exploit it first. But the second method means you're losing your weapons but instead gaining a shield.
As I see it, the shield is a lot bigger and has far higher utility. But part of that is that I see democracies as having differing vulnerabilities than autocracies. Attacking autocracies is more spear phishing, very directed attacks on the specific people that control power. But attacking democracies is in some sense easier (and in another sense harder) because more power is held by the average person. People who are more vulnerable to manipulation, especially at the large scale. But now we're edging into the data privacy domain and that's probably out of scope here.
I really think there should be a very strong blue team effort by these organizations. I am okay holding on to a specific vulnerability if you're going to attack a specific person in the ,,immediate'' future, but these agencies should also be working with companies to patch these vulnerabilities. That is the government providing a social good. You know, the reason we have the social contract and government in the first place.
Let me know the answer. Because I feel like that should definitely be part of it. Though there's some very concerning aspects of lack of defense for national infrastructure things like power grids. So I doubt it is being taken seriously, or as seriously as it should be.
I really do think a country should be proactively red teaming its own infrastructure and repairing any holes it finds. But it doesn't seem like the best interest of people who are more focused on offensive techniques.
The "disclose or weaponise" question gets very easy when, say, all of your adversaries are using Chinese software and all of your allies are using American software.
Also, I’ve got an email about any freshly imaged Mac Mini from Hetzner. Turns out macOS runs with legacy netbios ports open to the wide world by default, but to disable that service, you have to unload a service via Terminal. There’s no prefpane for that.
Got something similar here in the UK also. I once had a Linux server box running on my DMZ, got a few physical letters from my residential ISP (Virgin Media UK) saying they detected some open port that was recommended to be closed (Think it was NetBIOS port).
Might have been part of this scheme.
Don't have that box anymore (was around 5 years ago) or a PC on the DMZ so haven't received any since.
I doubt it. Network operators like Virgin have very good business reasons to ensure their own network isn't infested with computers running services like NetBIOS, which has no business being exposed on the internet (it is rather verbose, and completely useless outside of a LAN).
Does anyone remember that hacker that scanned printers and if they found a vulnerability they exploited it to print out a warning to the owner of said vulnerability? I think they patched it too?
That happend over 9000 times. Fun fact: Some are print server appliances, no patches or updates for some of those available as they are EOL - but still in use...
Yup. When I worked in “secret” level security, we’d often have an email circulation from “someone I can’t name” about potential vulnerabilities in software “I’m not allowed to talk about”.
> the best example is the UX of the main sites like car tax
To give you a comparison, in the US you need to go down to the DMV with a wad of forms, get the bits you can't fill in filled in, let someone make up a price, decide you haven't filled a bit in properly, send you away to a different window, get something else filled in, pay a fee for the filling in, hand the papers in at yet another window, pay for the actual registration, get a temporary registration slip, pay for a set of plates to actually be fixed onto the vehicle, pay for the stickers that say you've paid for a plate, all of which paid by cheque at various windows, with no real idea of the total cost up front.
In the UK (where cars tend to keep the registration number they're given on first registration), you go to the DVLA website, follow the prompts for the kind of paperwork you have (reminder letter, V5 registration certificate, V5C "green slip" if you've just bought it that the previous owner tears off the bottom of the V5 and gives you), it tells you how much it'll be per month or per year, you put your credit card details in, and that's it. Paid, done, nothing more to do.
Lots of this stuff is required to be done by US States, rather than by the US Federal Government, and Americans are in general keen to keep it that way, for a mix of good and bad reasons.
So every problem needs to be solved independently fifty times. People who live in continental Europe might have examples similarly because there are undoubtedly things European countries, especially EU member states could co-ordinate and don't. The difference in population between Luxembourg and Germany is even bigger than between Vermont and California.
I wonder how effective this is. The text suggests that the only thing that they look for is that they look for is a version statement of a major component, and then compare it to known vulnerable components. That could be somewhat helpful, but a lot of vulnerabilities won't be detected by that process. Does anyone know if they do more?
> What precautions and safety measures does the NCSC take when scanning?
> The NCSC is committed to conducting scanning activities in a safe and responsible manner. As such, all our probes are verified by a senior technical professional and tested in our own environment before use. We also limit how often we run scans to ensure we don’t risk disrupting the normal operation of systems.
That is it? So... One gal looks at in and says, yep, fire up the guns! All is go!? Can I see the publish test? Can I see your "own environment"?
What will they do if (when) this service gets whacked and delivers a DoS on a bunch of sites? Send the webmaster a free credit check?
How does cloudflare, akamai, and similar feel about this?
Personally, I would not trust my dog's toothbrush to any gouvernement.
I would be happy to see this if it was opt in.
In my opinion, they should have spent all the money on securing themselves, increase their own security education, increase security staff compensation to get higher caliber staff, and public education.
This is no different to Crawlers used in web-indexing projects (google, duckduckgo, baidu, Apple Siri).
You dislike this not because it could be defined as invasive (it's not), or because it could DoS websites (it won't). You dislike this because it's done by the government.
The gov spending money on this endeavour hardly bleeds the coffers dry nor does it prevent investment in other sectors. This kind of radical, tech-first thinking is the kind of thing we should appluad the governments IT service for doing, because not too long ago there was virtually no innovation, anything tech related was offloaded to incompetent contractors such as Accenture and Deloitte.
In fact you're contradicting yourself in multiple ways.
> "[should have spent money on] public education.": this can be defined as public education.
> "increase security staff compensation": as noted, not too long ago lots of this work was delegated to contractors. This is a step in the right direction and soon we can hope the compensation will increase.
> "increase their own security education": have you got a source which states software engineers working for the gov aren't educating themselves?
> "That is it? So... One gal looks at in and says, yep, fire up the guns!": What were you expecting, a full panel of industry experts scrutinising the code followed up with a parliamentary committee? This is called a peer code review, a rather simple process which you'll find at any tech firm.
There's a reality that the second you put something on the Internet it starts being scanned by bots and bad actors. The problem with the various "checking your door is locked" analogies is that you'd have to picture a locksmith putting in a door with five bad actors lined up behind up waiting to check it immediately.
The HN crowd is probably thinking about startups with in-house apps, but the canonical case for this argument is the Microsoft Exchange or Confluence servers that are consistently abused by ransomware actors - which then go on to cost that Government a lot of money as they get dragged into suppliers and contractors being held up for ransom, or investigatory resources being spent in response. It's very easy to tell who owns such a server, and being proactive helps address that.
It's fine to "not trust", but if this service poses any more of a threat to an organisation than the dozens of services already running you've got other issues.
Opt in would miss exactly the sites they're targeting. People who don't bother to even think about security let alone do anything about security since as long as it works for them they don't care what happens to anyone else or their data. Opt in would also miss anyone who has been setting up their servers/sites maliciously or acting as safe havens for crime for profit.
If these scans end up causing problems they should be on the hook for damages, but I'm glad they aren't waiting around for people to find them and reach out begging for scans. The responsible people keeping up with security issues and doing things proactively are rarely the problem and do their own scanning already.
I don't think their scans are more complicated than anyone could generate via metasploit, or worse a moderately good security engineer would likely run after pentesting your site for hours. If your site get whacked by non malicious intent payload, you should thank them that they prevented anyone fro directly starting with malicious intent.
more realistically id venture a guess this is a UK Gov agency trying desperately to justify its budget by getting overzealous with their Nessus scanning and then cowering behind the union jack when it comes time for public comment.
curious to know how long until the scanning source IP's wind up in my pihole.
I wish every government would try "desperately to justify its budget" by actually providing useful services for the people that make the global internet a better place for everyone.
Why not? Maybe they don't 100% trust their government for whatever reason. Why must they let their ports be subject to arbitrary inspection by an entity they don't trust if they don't have to? Hell, I'd block this port scanning on general principle.
I realize this is an emotional topic for some people but think about what this is really doing: the government isn't asking anyone to open anything up but rather scanning what they've already exposed to the entire internet. If there's something there you want to keep private, you should be closing it to everyone except the authenticated users you want to have access to it. You are not better off in any meaningful way if you block a public vulnerability scanner but leave yourself wide open to everyone else.
Consider, for example, the possibility that the government might have technical people at least as adept as the average teenager looking to pirate movies. If they were trying to something you consider malicious, would they a) put up a public web page telling you how to detect their traffic and stop it[1] or b) scan it from IPs which are not easily attributed? Using cheap commercial hosting for that would cost a fraction of what they pay a single employee per month and it's not exactly a technically-daunting task — and if it were, they'd toss a few thousand at Shodan.io to do it for them, an amount which could be buried in the printer supply budget of any national government.
> You are not better off in any meaningful way if you block a public vulnerability scanner but leave yourself wide open to everyone else.
That is a value judgment better made by the server owner, don't you think? It is their private (perhaps leased) property we're talking about after all. Perhaps the government should ask first before periodically scanning someone's property?
It’s privately owned but publicly accessible and there’s a strong public interest in shutting down insecure servers before they’re compromised by malicious actors. To me this seems more like the government having an inspector walking down the street and observing whether your building has broken windows, rats, and smells of gas. Any information they get is something you’re giving away to the entire internet anyway.
Again, I’m not saying you don’t have the right to block them - they even give you an easy way to opt-out - but that it seems misdirected to worry about the people asking nicely when the internet is full of actually malicious people who don’t ask.
> Why must they let their ports be subject to arbitrary inspection by an entity they don't trust
because that's literally how the internet works. Their ports are and will always be subject to arbitrary inspection while they are reachable on the internet.
The if they don't have to was a rather important part of the statement you quoted. I'm saying if they can block the scanning packets and have some subjective reason to do so then this is fine, the value of the scanning as a service notwithstanding.
The point is that if you want to have a device on the internet you will "have to" by definition. You don't get a choice. Everything on the global internet is subject to arbitrary inspection at any time by anyone. That's how it works. Someone may inspect your device and discover they are blocked, or ports aren't open, but arbitrary inspection isn't something you ever have a choice in.
Honestly my immediate thought was they were scanning them to hack them and keep the data private to use against you. Maybe it's just because I live in the US, but it's shocking to me a government would scan your server and not exploit that somehow and instead actually privately let you know what it found.
The US government has some similar efforts but they’re mostly focused on government agencies and what they consider critical infrastructure (e.g. the power company can ask for help, a donut shop is probably going to be told to buy a private service). It doesn’t make the news as much but domestic security is legally part of the mission for agencies like DHS and Commerce.
"As part of the NCSC’s mission to make the UK the safest place to live and do business online" those are pretty wildly disparate goals. Why would those two things be under the same agency at all?
> They live in the UK, and maybe spend time online
I think "maybe" drastically undersells the amount of time and things some people do online (generally the younger generations).
You're also overlooking that a very large portion of daily life has moved online and it's important to protect that. Everything from buying groceries, booking doctors appointments to looking up the menu of local restaurants.
I'd want all my personal and payment details protected, and it's reassuring to know information I'm reviewing hasn't been maliciously tampered with.
Given the percentage of global internet bandwidth that is video streaming, and the immense expense that entails, I find your >50% figure hard to believe.
"Valuable data can be kept for three days, and metadata for 30 days. One leaked document states that all metadata is usually kept: 'we pull in everything we see'."
"metadata" & "everything we see" is not the same thing as "all internet traffic". I'm not trying downplay the quantity or sensitivity of what is stored, its just the idea that all internet traffic is being recorded by any government seems technically very challenging and makes me very skeptical of that unqualified claim when I see it.
Taking responsibility for collecting and using vulnerabilty scan data in this case also means assuming authority to do so. A good test would be whether citizens are also free to inspect the vulnerabilities of government systems, or have a right to do so. If they don't, that's worth scrutinizing.
Canada has a different approach, where institutions can sign up to using a federal DNS service provided through the domain registrar, which I interpret is not unlike 1.1.1.1 or 9.9.9.9, but with malware detection. I believe it's called Canadian Shield, and it's not active scanning, but rather passive collection from institutions that manage infrastructure.
Active scans by government seems a bit like domestic intelligence collection. Given the techincal capabilities of most of these agencies when they work with ISPs, hairpinning traffic from one of these scanned servers for inspection is trivial. Fine if the threat model involved exceptional cases with clear oversight, and individual decision accountability in response to ticking bomb situations, but the examples of how similar powers have been used in the past are so abundant that I'm having trouble remembering a situation where they were used to protect a mere citizen.
I can personally attest to the fact that if your uninvited assessment of vulnerabilities reaches the level of gaining unauthorised access to computer systems - i.e. if you find something and check it works - you are technically in violation of the Computer Misuse Act 1990.
It's very easy to forget such laws exist because 99.99% of cybercrime goes unpunished - but that's for small victims, with hard-to-find attackers who are likely beyond the police's jurisdiction. If the 'victim' is an important government department, and you are within the police's jurisdiction, you could be one of the few people to actually face punishment - unjust though that may seem.
That's pretty cool. There are these pockets of really great public service internet services.
Am I interpreting correctly that you can join HackerOne to do work on UK public service projects? I tried to get something like that done for a municipality and a province, where it was going to be a way to engage college students on doing vulnerability hunting on public infrastructure, but also use it as a recruiting pipeline to get people interested in public service.
> Canada has a different approach, where institutions can sign up to using a federal DNS service
It seems far more invasive to route all your DNS traffic through a untrusted source than having that same source use the exact types of scans attackers are using every day already and report problems they find to you.
I can learn a hell of a lot more about you by your DNS history than I can from knowing what ports you have open and what vulnerable services you're running.
"where institutions can sign up to using a federal DNS service provided through the domain registrar..."
The domain registrar is CIRA, and has only one of twelve board members having a federal government affiliation. See cira.ca for the facts. Their Canadian Shield services uses data from Akamai, Mozilla, and CCCS.
It is not "federal".
Sigh. Another comment from someone's memory that takes only 2 minutes to fact-check and discover to be incorrect.
Would you like to elaborate on who collects the data and does the security analysis on it and offers to provide threat intelligence back to the institutions? I can save you a click:
It's a useful service that I was commenting on as analogous to the one being provided in the UK, and it taking a different and passive approach. Instead of apologizing, my favourite charity can be found at victimsofcommunism\.org
waiting for india to implement something similar for seemingly benign reasons like vulnerability and code quality and immediately use it to find critics and hang them.
heck, a guy was sentenced for 5 years over a facebook post.
>We design our requests to collect the smallest amount of technical information required to validate the presence/version and/or vulnerability of a piece of software. We also design requests to limit the amount of personal data within the response. In the unlikely event that we do discover information that is personal or otherwise sensitive, we take steps to remove the data and prevent it from being captured again in the future.
what is preventing a government to disregard the removal of sensitive data? why can they not weaponize this?
>In the unlikely event that we do discover information that is personal or otherwise sensitive, we take steps to remove the data and prevent it from being captured again in the future.
beyond a promise, what assurances do you get it wont be weaponzed?
Anyone who has worked with Chinese companies operating within China can tell you that very similar laws were enacted a year ago. The CCP has a law that any vulnerabilities made aware to private companies need to be disclosed to the federal government. This was done in the name of "national security". IMO, this seems to be a more veiled version of that same mindset.
I was about to say how great I think that law is, but then I checked the link you provided...
> anything discovered in the country must now be reported to the CCP *and to no one else* (in most cases).
The "no one else" part is terrible and completely changes the story. However, I do generally support a "tell the government about discovered vulnerabilities" law. Ideally, the government would then inform affected users and investigate whether the vuln could be considered negligence and the company prosecuted.
I've been in a few situations where I reported very easily exploitable vulns that leaked sensitive user data and in all cases, I couldn't for the life of me convince the companies to disclose the leak. Yes, I could've gone public myself where I didn't have a contract, but I would've 100% ended up in jail for some poorly defined crime of "hacking".
[1] - https://www.shodan.io/