As a security professional I'm very disappointed when I read the (often angry) messages people have written to the maintainer/author of this software. You can read them here, in the ip range blacklist: https://github.com/robertdavidgraham/masscan/blob/master/dat...
If you don't want people on the internet to connect to your server, then you shouldn't allow network connections to your server. A few connections per minute hardly classify as abuse in any reasonable sense of the word.
The owner of the source network, on the other hand, (compared to the destination server), I think have a more legitimate reason to raise flags. Most cloud providers for example do not allow port scanning from their network (for various reasons). If you're gonna send out millions of packages all over the place I think it's good practice to inform your network provider first.
Yes this! I'm not a security professional but System Administrator, and i need tools like that, especially DoS and metasploit-kind tools to harden my own systems/services. We need tools like that, if we don't try to break our own Systems, someone else will do it (and this time with consequences).
BTW: Massscan is excellent at braking routers, over-flood them and they will often crash.
As another security professional, I'm laughing my ass off. Imagine wondering what targets might be nice, and reading the comments in this file. How nice of them to email and tell us exactly which IP ranges are sensitive!
Not everything is a technical system, and technology does not exist in a void. I'd think the field that coined the term social engineering would understand that.
The tech world is in for a rude awakening as technology becomes less a field of nothing but specialists, and actually gets infiltrated by a greater and greater number of tech-savvy, yet industry independent stakeholders.
We've had a social blank check to work from for the better part of half a century. You now have people writing children's intros to k8's. If you don't think that at some points technical problems don't start getting solved via social/legislative/legal means, you're in for a bit of a rough time.
I'd guess they'll take our their home connection's networking kit at 10M pps before they take out the target server(s). Tools like massscan generally send 1-2 packets per port, so a target host just has to deal with < 130k packets, even assuming no intermediate firewalls are dropping the traffic to unused ports.
Also, minor nit, if it's just one machine doing the scanning, that's a DoS, not a DDoS :)
Then u send 1-2 packets to apples entire ip range, there's not much harm done, the tool doesn't send packets to one and another, it partitions the whole ip list and sends it in random order so that not a single endpoint gets hirt hard , usually a single machine can handle easily 100k packets per second, they not hitting an application most of the time at all, they incomplete tcp packets, and just check for existence, they not sending a a huge chunk of packets repeatedly
I am not sure I would trust sending 16,777,216 packets to Apple several times in a row. Especially a company with a legal team as large as apple.
What is the difference between this tool and the drive by DDoS "testing" tools you can pay to use online. They seem identical to this tool except Masscan stops after the first try.
I'm sure ur already sending 10k+ packets from just casually browsing their website, albeit they would be a lot bigger than massscans(which is a couple of bytes), vs http (a couple of million bytes(assuming the website isn't plain teyt)
They're also not apple :) These companies are probably just getting alerts from their intrusion detection system telling them someone's port scanning them.
I'll amend my earlier comment a bit. Apple as a corporation doesn't care, neither I imagine would Apple's corporate IT security as an entity, but individual people in Apple might decide port scanning is the bain of their existence and send something, but that's a fluke.
The difference this this tool is a hammer and like any tool the operator is responsible to use it safely and appropriately, while the botters are people advertising that they got a hammer, and are willing and eager to bludgeon people to death with it for some money.
Apple wouldn’t even notice. For example recently there was an article here about 5 hackers that spent 3 months attacking Apple (white hat) and Apple seemed unaware until the bug reports were sent in.
Ddos tools usually use amplification, instead of sending 1:1 bytes
(that is u sending 1 byte and receive 1 byte as answer)
They may query a database instead which a 30 byte search query results in couple thousand bytes of results + the load on the database
It would be expensive to just use raw network power to overwhelm a web service(u would need more bandwidth than the host)
Meanwhile with amplification u only need a 10th or less
Whilst I have no inside information on Apple, I'm pretty sure that'd be a tiny portion of the traffic they see daily. They may blackhole you just to cut down the noise, but frankly given the level of DDoS they'll get regularly, I doubt they'd bother.
For one very quick stat "The average size of DDoS attacks was at the mindblowing 26.37 GBps in Q2 2018"
You got your units wrong. 26 GBps (208 Gbps) would be among the largest attacks recorded in history. Maybe you meant to say 26 Gbps which is 8 times less, but even that is a very large and notable attack, hardly any company could withstand it outside of CDN and big tech.
that's a direct copy/paste from the first article on DDoS size I saw https://hostingtribunal.com/blog/ddos-statistics/ , I wasn't going for deep research, just making a point about massscan from one host not really representing a serious concern for someone like Apple.
If you have something listening on a port that can fall over by someone opening and closing a TCP connection, maybe that special something shouldn't be listening for things on the internet.
> A few connections per minute hardly classify as abuse in any reasonable sense of the word.
That's for one individual who's scanning something. On the receiving end, you're not dealing with one individual, you're dealing with many individuals who are probing for vulnerabilities.
If one guy intentionally steps on your foot, that's mildly annoying. If a thousand people intentionally step on your foot, that's a very different issue.
> Most cloud providers for example do not allow port scanning from their network (for various reasons).
They don't? They are often the source I see. Is that a policy thing where they say "yeah well please don't" or will they actively shut you down if you're doing it from their infrastructure?
Cloud providers are vigilant for signs that accounts have been compromised and are being used by hackers for nefarious things. It's in their best interest to detect this early and step in before the hackers can pile up a ton of charges that the account owner is then going to dispute. Some big clues are (1) seems to be mining cryptocurrency, (2) seems to be trying to DDoS something, (3) seems to be sending email spam, and (4) seems to be scanning the entire internet for vulnerabilities. Sending a ton of email is usually actively prohibited and the cloud provider will blackhole your packets because they're protective of the reputation of their address blocks. Scanning the internet is more of a "try it and see" sort of thing. If it's not a significant change in behavior from the background of what's normally going on in your account, or if you're doing it from a trivial number of machines, probably nothing will happen. If you suddenly spin up a ton of infrastructure for this purpose you can probably expect a friendly phone call fairly quickly, followed by having your account suspended until they hear back from you. If you run a big account with your cloud provider they won't go suspending your VMs willy-nilly, but also if you have a big account with your cloud provider they have your business number and expect you to answer it.
I once ran nmap from one EC2 in our account to another via EIPs (ie out to the internet and back again) to test the firewall and got a nastygram from AWS about running scans.
Maybe I'm misunderstanding but how is this blocklist supposed to block anything. Take for example I'm a hacker that wants to port scan general dynamics (they were first on that list). Wouldn't all I need to do is remove their entry from the config file?
Well yes, but it's not there to stop you, it's there to stop people who are smart enough to use masscan, but not smart enough to compile it. And I guess, much like locking your front door, there's also an element of keeping honest people honest.
> yes, configuration files are specified on the command-line and not hard-coded, so only those performing legitimate surveys of the Internet (possibly wanting to be responsible or respectful of those NOCs who still live in the world of generating abuse complaints when snort tells them to) would be likely to use them. Maybe there are a few script kids out there who are intelligent enough to avoid hitting the small collection of networks on this list to avoid their scans generating abuse complaints that may get their boxes killed, but I guess it's probably a near-zero population
Yes. I suppose, though, it's a win-win situation for both parties because the author can claim to have addressed complaints and hollow threats while allowing anyone to do as they desire and, may we say irresponsibly, remove it.
Also a win for a third party malicious actor - they get a list of networks where the administrators decided to try to block scans instead of addressing their own issues...
I don't really understand the author's stance to be honest. I agree that it's silly to blame them but at the same time the author explicitly acknowledges that the tool is meant to mass-scan the internet and that it's a bad thing:
>While useful for smaller, internal networks, the program is really designed with the entire Internet in mind. It might look something like this: [...]
>Scanning the entire Internet is bad. For one thing, parts of the Internet react badly to being scanned. For another thing, some sites track scans and add you to a ban list, which will get you firewalled from useful parts of the Internet. Therefore, you want to exclude a lot of ranges. To blacklist or exclude ranges, you want to use the following syntax: [...]
That 2nd bit is where that "exclude.txt" file comes in, it's not even used by default as far as I can tell.
So basically the author acknowledge that the software's intended purpose is bad, they also decided that it was their responsibility to maintain an exclude list. That's a bit odd IMO. I'd think that in these situations you can either say "I'm not responsible for people misusing my software" and in this case maintaining an exclude file with random addresses as people complain to you doesn't make sense, or you think that you share some of the responsibility if your software is used to do bad things and then it seems like it would make more sense to take the project down or take steps to make it harder for users to do these things.
The issue is that in large enough population of people there will always be some that just don't understand the issue.
The people who look to project github for help are ones that already selected themselves. I bet for every one that posted on github there are dozens or more that went to the actual entity that tried to scan them or, better yet, blocked the scan or otherwise ensured it is harmless.
I wonder how often the reverse happens, i.e. admins sending angry emails to their own users who appear to be scanning other people's networks.
Anecdote: when I was a grad student at AS88, I once got an email asking me to stop port scanning. I was confused because I wasn't port scanning anyone. I asked for details and an admin sent me a report generated by some seemingly off-the-shelf network admin software (forgot the brand), with a bar chart of all IP addresses I was frequenting -- rather creepy, honestly. Turns out I was renting ~20 servers around the world as PoPs for a personal project at that time, and regularly deploying code to all of them at once over SSH (all configured at port 22). Apparently regularly accessing ~20 servers at once over a single port was enough to be flagged as "port scanning". I wonder if people doing actual security research over at the CS department were exempt from nonsense like this.
> How about simply renaming curl to zurl so that you end up at the very bottom of the list and hand over the case to the next dependency in alphabetical order?
that one's amusing, but this one, in a git repo, is :chefs_kiss:
#NOTICE: This e-mail and any attachments is intended only for use by the add=
#ressee(s) named herein and may contain legally privileged, proprietary or c=
#onfidential information. If you are not the intended recipient of this e-ma=
#il, you are hereby notified that any dissemination, distribution or copying=
# of this email, and any attachments thereto, is strictly prohibited. If you=
# receive this email in error please immediately notify me via reply email o=
#r at (800) 927-9800 and permanently delete the original copy and any copy o=
#f any e-mail, and any printout.
Maybe we should call them, to say we "averted our eyes, m'lord"?
I think that if people are upset their name is up on a list clearly meant to shame them for being stupid, then contacting the author to say "ok I understand now, can you please remove us from that list" will definitely get them removed.
But I suspect if someone is the kind of person to email a tool author because of what an unrelated tool user is doing, they're probably going to be quite chuffed there's a tangible outcome they can point at and say "see, I fixed the issue"
#Received: from elbmasnwh002.us-ct-eb01.gdeb.com ([153.11.13.41]
# helo=ebsmtp.gdeb.com) by mx1.gd-ms.com with esmtp (Exim 4.76) (envelope-from
# <bmandes@gdeb.com>) id 1VS55c-0004qL-0F for support@erratasec.com; Fri, 04
# Oct 2013 09:06:40 -0400
#To: <support@erratasec.com>
#CC: <ebsoc@gdeb.com>
#Subject: Scanning and Probing our network
#From: Robert Mandes <bmandes@gdeb.com>
#Date: Fri, 4 Oct 2013 09:06:36 -0400
#
#Stop scanning and probing our network, 153.11.0.0/16. We are a defense
#contractor and report to Federal law enforcement authorities when scans
#and probes are directed at our network. I assume you don't want to be
#part of that report. Please permanently remove our network range from
#your current and future research.
#
#Thank you
#
#Robert Mandes
#Information Security Officer
#General Dynamics
#Electric Boat
#
#C 860-625-0605
#P 860-433-1553
You would hope a defense contractor was smarter than this but of course they tend not to be... threatening to put a maintainer in a report to Federal law enforcement is weak sauce.
If you own a building with many doors and you catch someone spying on the doors trying to find one left open, it’s not much use to complain to the manufacturer of the binoculars.
Well scanning has some useful security research applications. The part about scanning the whole internet is useful if you want to do larger scale research (like how many % of the internet run telnet).
In addition massscan is really interesting from a software engineering persepective. They do kernel bypass to talk to network drivers directly, have a custom TCP stack, custom mutexes, etc. All of that to be able to reach ~1.5M packets per second (from the README), allowing someone to scan the whole IPv4 range in 6 minutes. Really impressive work.
There's an argument to be made that if your system accepts a connection, then that's permission to connect.
If it then accepts a HTTP Request, as these folks have, then that's permission to continue with the request.
If they didn't want to accept any connections, they should've closed the port.
If they only wanted authorised users to connect, they should put an authorisation requirement on the connection.
As for your statement that there's no valid research purpose, that's your assertion, a lot of people will disagree.
What a machine does in response to a network packet does not prove the intent of the owner of the system. Otherwise RCEs and local privilege escalations would be permission to do as you will to any system which us susceptible to them.
Is it valid to research untreated syphilis in black men? Yes. Is it valid to research untreated syphilis in black men without their permission and informed consent? No.
That's pretty much a text-book straw-man argument. Two of them, actually.
We're not discussing the morals of human research, we're discussing IT security, and specifically in scenarios that are unlikely in the extreme to lead to any physical harm to people.
We're also not talking about trying to sabotage or attack a system.
Standard, well-formed web requests on standard HTTP sockets, made infrequently are very unlikely to cause problems on any system that's not outright deliberately misconfigured.
> Is it valid to research untreated syphilis in black men without their permission and informed consent? No.
The Tuskegee Syphilis Study was inappropriate because they purposely withheld information about the diagnosis and lied about the efficacy of the treatments they were giving.
The problem wasn't that they conducted the research without telling the patients. Obviously the patients were aware that they were part of a research effort, since they directly interacted with the clinicians conducting it.
This isn't exploiting a vulnerability. If I knock on your door and you open it and invite me in, you don't get to complain about how I'm trespassing. If your server responds to traffic on a port, then it's completely fair to conclude that you're okay with that.
Then upon them discovering that it's open to the world, their first response shouldn't be to blame the people connecting, but themselves for leaving it open.
Huge numbers of people have done this, I've certainly done it in the past when I knew less.
Discovering that the webserver logfiles were rather larger than I expected, or that HTTP Traffic was through the roof... oh, right, I left something open, better close that and remember it for next time.
Just chiming in from the other sub-thread to say that I actually agree with you on this point. A misconfigured access control policy isn't grounds to assume that you have access.
However I don't think it's a reasonable assumption that the open status of TCP ports is supposed to be private information.
It's valid to perform research on Earth by taking satellite photos and examining them for land cover use. It's valid to use a laser range finder to map the land elevation of the entire planet. Among the various governments which have done this, Airbus has as well. Presumably to ensure nobody flies a plane into a mountain.
If it's valid to shine a laser pulse from space/the sky onto the entire surface of the Earth (private property included) to determine the elevation of everything, it's valid to send TCP SYNs out to the entire surface of the internet.
That's a poor analogy. The outside colors of the majority of the homes in your neighborhood effectively broadcast that information into the public sphere. A more analogous question is "Is it valid to conduct research about all the homes in the world by around and attempting to peer into all their windows to see if there are any interior bedrooms with pink walls and fluffy animals?"
Alternately, you could ask me "If a computer is broadcasting packets to my computer which is connected to a network port that I'm allowed to connect to, is it ok for me to note that?" In which case I'd say yes.
Scanning through windows to find fluffy animals is certainly getting closer to crossing an ethical boundary but it's not strictly illegal.
I also think that moves the analogy too far in the other direction. While it may not be the case that servers are actively "broadcasting" their port open status to all internet users, that's only because it would not be feasible to do so. It is not because port open status is supposed to be private or secret information, like the inside of my bedroom for example.
In fact TCP/IP is designed with the intention that anyone can check the port open status without authentication. It is part of the intended usage of the protocol, so presumably it is safe to assume that anyone using it is probably using it with that intention.
Wut? It's like saying that going around in a google car taking pictures of all doors has "no valid research purpose". (And yes, checking if the doors are opened or closed, or cataloguing their locks, would be fairly legitimate too; as long as one does not step into an actual property, at which point it's trespass.)
Scanning for open ports sounds like a legitimate use, but being a Network admin and not capable to block a robot called "masscan/1.0" is NOT legitimate.
Scanning for open ports on other peoples networks? How is that different from casing a house for entrances? It’s behavior only two types of people would have: criminals and security professionals.
> Scanning for open ports on other peoples networks? How is that different from casing a house for entrances?
It's more like looking through other people's windows as you walk by the street. May be creepy if you always stare at the same window, but formally there's nothing wrong with that. If you do not want people to see through your public-facing windows in your home, it's your responsibility to install blinds or shades.
If you walk around town looking in every window, you’re probably not gonna have a good time when you get detained or shot. Let’s not pretend like there are legitimate use cases for scanning ports of networks you don’t control.
> If you walk around town looking in every window, you’re probably not gonna have a good time when you get detained or shot.
Dude, what kind of town do you live in? It sounds scary!
Here in Europe many towns have narrow streets with houses directly by the street (with no front yard). It is essentially impossible to not look through the windows of people unless you make a robotic effort to avoid it.
TIL there are hackers living in high-crime areas. Seems like a good idea to move to a neighborhood where you won’t get shot for having your eyes up, which is almost all of my city.
A few years ago, in Paris, my commuting by train got perfectly synchronized with the alarm clock of an old gentleman who lived next to my workplace. Every day, he opened his window exactly as I was passing in front of it. After a few weeks, we got to know each other in that strange way that people "know" each other but have never talked. After a few months, we were saying "bonjour" to each other, and sharing a smile. I would be honestly surprised if the man suddenly decided to shot me! Would that be a common thing to do in the U.S.A. ?
Don’t be daft. You weren’t peeking in his window or at his back door, and you didn’t do that to a thousand houses. If you think this is acceptable, I dare you to go around the nearest city to you and start checking all the windows and doors on every house you can for a day.
If you don't want anyone connecting to you network, don't connect it to the internet. If you only want certain people to access your network, whitelist their IPs.
If you don't want people to look at you, don't go out in public.
Or just curious people who interact with the internet directly for fun. Not everyone is confined to a web browser and facebook. No one is hurt, no one is defrauded, and no one's privacy is impacted by using the internet as it is meant to be used.
> How is that different from casing a house for entrances?
If you don't see a difference then I hope we don't share neighborhood.
> It’s behavior only two types of people would have: criminals and security professionals.
consider also curious people seeking knowledge; the Internet is a massive space and an interesting phenomenon in itself, scanning is one of many ways to learn about it.
Hehe, and the experience comes from books...no need to do it with your own hands. Same with pilots just read a book and that's it...maybe some Microsoft flight-simulator hours too.
This tool is like a flashlight for the Internet. It reveals the location of all the cobwebs. At the same time, there are admins out there that don't like you shining lights in their windows and they will ban you for it.
Everyone knows flashlights are used by bank robbers. But if you can think of a legitimate use of a flashlight, then you can think of a legitimate use for this tool.
I am wondering the same thing myself; it's not obvious to those of us not looking for vulnerabilities in other people's systems what the use of it is at internet scale.
Flashlights are used by normal people for strictly local, very closely defined location illumination. We don't turn them on and light up the whole world.
I’m working at a cyber security company, we have an agent running on customer machines that helps identifying security breaches.
One tool we use is to scan the customer’s network and find out which machines don’t have our agent running.
That you probe your customer's networks, I can understand, but, but why would you scan other people's networks?
I encountered a bunch, always trying the same, lame URLs for a PHP framework bug or a Wordpress config error, sometimes a hundred variations in a few seconds. What's the point? Same goes for all these ssh connections, but they seem to be real hacking attemps.
I feel like most of us who rolled our eyes at all of the "cyber" stuff have generally thrown in the towel. I see it mostly used when talking about security-related issues as opposed to just "on the internet" stuff nowadays. Cyber-warfare, cyber-security, cyber-espionage, etc. I do still chuckle a little bit, but language and popular terminology is just funny like that.
Scan any large network instead of the whole internet. If you want to scan many ports in a large network quickly then you need an "internet-scale" portscanner like masscan.
"This is an Internet-scale port scanner. It can scan the entire Internet in under 6 minutes, transmitting 10 million packets per second, from a single machine."
Then it seems that it has a link to its github page by default in the User-Agent string it uses while scanning.
When you do that you can only get abuse in return, can't you?
If someone's scanning the internet at that speed, that means you get less than one packet per five minutes for each IPv4 you have. Hardly something to whine about.
If one person in the world is scanning, yes. Ten persons => 1 packet every 30 seconds. 100 000 persons => 1 packet every 3ms. Suddenly doesn't look that innocent, right?
So in this situation where you have an absolutely ridiculous number of people flooding the internet with scans, all on multi-gigabit connections, it eats up a whole 20 kilobytes per second.
10 000 persons doing this simultaneously is also an insane number, and that's 2 kilobytes per second.
Mine got disconnected for about 30 minutes. Then I just calculated the rate I needed for the duration I found acceptable (one week for a single country) and let it run at this rate; no issue then.
i'd say "the internet" is the biggest "network of networks" and ipv6 is not really a part but a separate network ie. not interoperable with "the internet". maybe it will be more important some day, but not today.
If you are running this tool, you are sending packets from a single machine to N remote hosts, so logically the strain and bottleneck should be at your end and I can't see how this should cause much of a burden several hops away.
At the destination this should be a small fraction of the usual "Internet background noise" which is usually a negligible fraction of the available bandwidth.
You can separate transmit and receive IPs, if your tx network does not implement source filtering.
So you can tx from one place and receive from one or more other places "sensors" that you use to receive SYN-ACKs.
You can use several (an arbitrary number) of spoofed source IPs on tx to hide your "real" rx IPs, at the cost of more egress traffic.
There is a technique involving ipids (idle scanning) you can use which does not reveal your IP at all but it is not reliable; read: not usable beyond very tiny scale. You could put a lot of effort into it but it's not worth it. Nobody beyond a few vociferous cranks _really_ cares about IP scanning.
The real way to stay off radars (eg dshield) while mass scanning is have a ton of unrelated IPs and scan as slow as you can stand. This assumes good randomization (not obviously striping across networks from the same IP).
Individual users running massscan et al, are not going to produce anything like the level of traffic all the major providers (and quite a few sitest that aren't that large) see from DDoS attacks on a pretty regular basis.
Given that there are several sites who scan the Internet regularly for more than just open ports (e.g. Shodan, Binary Edge, Censys) it's not a volume of traffic that should cause a concern.
People who don't change the default user agent are legitimate security researchers. People with actual malicious intent change it to whatever the current Chrome UA is.
Probably not a great idea to make it so easy to scan the entire internet. Even providing the command in the readme before explaining why it is a bad idea.
There are numerous public domain tools that do this, so the cat is already out of the bag.
Last I read about it, you can scan the entire IPv4 space for a port in about 40 minutes providing you have the bandwidth and a forgiving ISP. I see another comment claiming a tool can do it in 6 minutes. Easy at "apt get install" and a single command.
Note that "scan the internet in 6 minutes" only means this tool is capable of generating packets fast enough on the host machine to theoretically do a 6 minute scan. In practice, the NIC, home network, and local ISP connection will bottleneck and the scan will be orders of magnitude slower.
It's been trivial to do for ages even before masscan existed.
Even then, you exist as an entity on the internet to have things connect to you. If there are ways in which you don't want to be connected to, you have a firewall to enforce that.
If you don't want people on the internet to connect to your server, then you shouldn't allow network connections to your server. A few connections per minute hardly classify as abuse in any reasonable sense of the word.
The owner of the source network, on the other hand, (compared to the destination server), I think have a more legitimate reason to raise flags. Most cloud providers for example do not allow port scanning from their network (for various reasons). If you're gonna send out millions of packages all over the place I think it's good practice to inform your network provider first.