When he explains how Perl handles an array passed as an argument to a function:
my @list = ('b', 'c');
test('a', @list, 'd');
Which ends up flatting the @list array into the argument list and printing 'a', 'b', 'c', most Perl programmers defend the language saying that the author just doesn't understand Perl, and that's how things are done in the language since forever. The response is basically a RTFM.
If you really wanted to send the whole list as single argument to the function, you should have sent an array reference, like this:
test('a', \@list, 'd');
Perl programmers say that if you took a minute to understand the language, you should have known this. However, what they don't realize is that Perl gives you ambiguous hints about this all the time. For example, the built in push function doesn't seem to follow this logic:
push(@list, "item2", "item2");
This function knows perfectly that the list is a single argument and the other ones are the items to push into the list. You don't have to pass a reference to the array to make it work.
The push function works like this because it uses an obscure feature called function prototypes. Which is rarely mentioned in any tutorial.
When a Perl newbie sees the push function, it's more than reasonable to think that passing arrays as function arguments will work as in any other language. When they find what really happens, it can be very frustrating, specially because Perl won't give you any error whatsoever. This is one of the reasons why Perl is so difficult to learn.
Granted, and most people think prototypes are evil for similar reasons. Perl doesn't have the same design goals as something like Python, it will accept inconsistencies and complexity if they're deemed useful
I personally like this, assuming the complexities aren't too great and the documentation is good but to each their own
Edit: In Perl land it is a bad idea to treat built ins as being identical to subs, and this was taught to me by the books I learnt from. The rub of this is that it is bad to generalise from stuff like print, push etc
As an ex Perl guy (well I still use it sometimes), I find append and extend an endless source of confusion in Python. Especially as strings do the opposite of what I expect, and get treated as character arrays with extend (or was it append).
I like the way that in Perl you have to explicitly reference the array, as this leaves no ambiguity about what is happening. I guess its just what I am used to.
"As an ex Perl guy (well I still use it sometimes), I find append and extend an endless source of confusion in Python. Especially as strings do the opposite of what I expect, and get treated as character arrays with extend (or was it append)."
list.extend(data) works like "for item in data: list.append(item)"
strings (str) do not behave differently than a list or a set. Try .append() on a list or a set, you get the same behaviour.
Maybe your confusion comes from the fact that strings are immutable in Python, and a list is not a string.
You may like bytearray() in Python 2 (bytearray.append raises an error if you pass a string longer than 1 character), but there is not mutable type for Unicode strings.
Yes but I would expect my strings to behave as a scalar variable, not a list or set.
I prefer the Perl way of having one function push, and specifically referencing the list if you want it stored that way. That has caused me far less confusion in Perl than the Python way of doing things.
> The push function works like this because it uses an obscure feature called function prototypes. Which is rarely mentioned in any tutorial.
I think the push function behaves differently mainly because it's a builtin. It is possible to use function prototypes to make your functions act like builtins, but the reason that isn't mentioned in tutorials is that most Perl programmers are less than impressed with prototypes. Except when passing code blocks as subroutine references, since prototypes allow you to omit the comma after the code block.
I think push is better understood if you realize its an operator. At a higher level what is happening is you are adding "item2" (and another "item2") to the array @list. ie @list += ("item2", "item2")
That's a fair point and it was certainly true that this tripped me up when I was learning Perl too. But now I've wrapped my head around the concept, I love it.
I love how he is building his own SQL string and then blaming Perl's DBI->quote for any vulnerabilities that arise. Anyone who writes SQL like that is writing bad code from the offset (regardless of the programming language nor it's DB/web frameworks). Parametrised queries and ORMs exist to prevent the kind of SQL injection attacks he's demonstrating and Perl's various DBD modules already support parametrised queries (in fact most Perl DBI guides will walk you through using them!).
This is pretty much "working with databases 101" and if he can't get even that much right then he should not be stood in front of people lecturing about the evils of any language nor it's framework.
Sadly though, these days proper security research is less important than looking cool in front of a small crowd. After all, who needs to have any knowledge about your subject if you can curse a little and display a few slides of some tired old internet memes. :/
I'd say that the problem has more to do with SQL and respective APIs than anything else. The fact that SQL injection happens so freaking often despite the fact that it's pretty simple to avoid should speak volumes.
I prefer ORMs, but they're a lot of overhead to get set-up so I don't blame others for using strings where appropriate... and every parametrized query API I've ever used has been a tremendous PITA which is why people throw up their hands and say "screw it, I'll just escape the quotes!". And of course, the database API could provide a good method to do that but nobody ever does because it's impossible to do it securely (and that's definitely a fault in SQL).
Have you used parameterized queries in Perl? It's the same DBI API and only about 3 or 4 additional lines. Plus you can easily abstract that away into a little helper function if you desired.
I'm on my phone at the moment but I'd be happy to post some example code later if you want it?
The following won't break (at least not in the bad way), I don't think:
$sth = $dbh->prepare("SELECT document FROM table WHERE tag=? AND security_level=?");
$sth->bind_param(1,foo());
$sth->bind_param(2, $user_level);
$sth->execute();
Ah yes, but that's a slightly different feature to the one demonstrated by the author (https://www.youtube.com/watch?feature=player_detailpage&v=gw...). However that's not to say that the vulnerability you raised isn't also a serious one developers need to be mindful of.
I love how, at the very end of the talk, when someone asked the speaker if he knew of any other languages that had this problem, and the speaker didn't know, someone decided to shout out "PHP!"
There is something that PHP developers should be made aware of, but it isn't something as weird as Perl's list behavior.
If you access /test.php?a=1&a=2 you will be greeted with:
array(1) {
["a"]=>
string(1) "2"
}
You can reason amongst yourselves about whether 1 or 2 is more desirable, but it's only one parameter. So the Perl list trickery is not applicable. Now visit /test.php?a[]=1&a[]=2
If you're expecting a string in a GET/POST parameter, make sure you're not receiving an array. This is a HTTP feature, not a bug, but it can sometimes break code if you're not expecting it. (Break as in error messages)
So, no, PHP is not an example of a language with a similar vulnerability.
I'd be fine with the swearing if it'd be backed up by some knowledge and experience, but that smug attitude is just incredibly irritating when coupled with such level of ignorance.
First decide what are you going to bash. Some libraries, some applications, the entire programming environment ?
Using induction from DBI->quote/CGI and a few old apps to bash the entire Perl ecosystem is just boringly stupid.
It is fine and useful to patch those old apps, but to give an arrogant lecture as if all programmers use the same broken approaches in 2014 is just ignorant.
It is extensively documented to prefer placeholders when working with DBI, that is, if you even use DBI directly and not via an OO mapper.
CGI as an approach is entirely deprecated on all languages/platforms not just Perl.
List expansion ? It is a feature, use it or leave it. You have the option to use references.
If you prefer Python, just use Python. It's like bashing C for having pointers.
So, what else ? Try PHP with those examples. Find some Ruby apps prone to sql injection. Bash some NodeJS libraries.
If Perl is dead already, be a rockstar bashing the new stuff, why even bother with the ghosts ?
I think it is because deep down all Pythonistas know that Perl is a far superior platform and they all carry a deep ancestral envy.
Otherwise they'd just be happy with their choice already. :)
This guy just comes off as very smug about a language he apparently has no idea what he's doing in.
Bascially all the WAT?s are because Perl's lists are very different from Python's. If he had spent five minutes reading the very first chapters of Programming Perl he would make much less fool of himself. He explicitly asks to flatten the lists, so there shouldn't be much surprise that Perl obliges.
That said, the security issues he pointed out are real ones. Be very careful in you are programming CGI (in any language)!
First of, I'm not a Pythonist. Python has nothing to do with the faults of Perl - Perl does.
Perl was a great language back at the 90's and early 2000 - it doesn't now. That is mainly because of the 'write-only' code style and many not-that-intuitive behaviors other languages practice better.
It is true that my talk was based on bad programming practices, but that's the practices actually used in the wild - in Bugzilla, TWiki, MovableType, and I haven't even started talking about what cPanel developers did.
So, if 99% of programmers doesn't use the language properly, who's fault is that? A company needs to provide its costumers with a working, intuitive product. So is a programming language. If so many people doesn't know how to use it properly, I'm sorry, but it's the language fault.
Another thing pointed out is prepare(). Yes, you can use it, no SQLIs there. BUT, this has nothing to do with the point. The point is that list expansion behavior in function calls is a problem most programmers weren't aware of.
These are some code sections used at Bugzilla 4.4.6 (fully patched as of 30/12/2014) at different places:
$attachment->_check_content_type($cgi->param('content_type'));
$cgi->uploadInfo($cgi->param('data'))->{'Content-Type'};
IsValidQueryType($cgi->param('query_type'))
And it also has 2 different quote() occurrences:
$dbh->quote($cgi->param('requester'));
$dbh->quote($cgi->param('requestee'));
Now, I'm not saying all of those leads to a vulnerability, but as you can see there's a very visible trend here.
Did all of those programmers and maintainers never read the tutorial for the language? Or did the language documentation confused them to the point they simply weren't aware of this behavior?
From a personal perspective of one that did try to figure out what's going on with lists I can definitely say that this behavior is documents - At several places, very differently. As a simple example go back to my slides and look at the CGI documentation screen shot. VERY confusing, and honestly, just false.
As for the attitude, I do apologize for everyone offended by the young douchebag that attacked your language. But, as recent vulnerabilities showed us, without a proper show no one's gonna notice you and your point, important as it may be. So, yes, I added a couple of funny images and built a momentum for a punch line, but otherwise this talk wouldn't have got the buzz it's getting right now and programmers would've still be blind to this behaviors, as sad as it may be.
For an ending note, I do believe Perl has done it course, especially for large, maintained systems such as Bugzilla and cPanel. It may be the end of an era, but who said this is such a bad thing?
Thank you for your comments (positive or negative) and for reading this.
Excellent, you're here so i (we) can address a couple of the points in your talk.
1) DBI is not a core module, nor to my knowledge has it ever been.
2) I'm the current maintainer of CGI.pm - i am not the author. This is an important point because the module is 20 years old, and like any software of significant size/age/importance it has been through several different hands (over 30 according to the git log, which only goes back to 1998 so is missing 3 years).
It would be great if i could just release a version of CGI.pm that removes the list context behaviour of ->param but that would massively break back compatibility for hundreds, if not thousands, of users. Not all of these users have the knowledge or resources to fix their code, which is why for the time being CGI.pm will warn when ->param is called in list context. This has actually broken some software (anyone that has set warnings to be fatal), but this was the least harmful way i could get out to users who maybe blind to the issue that there is indeed an issue.
And yes, as many have already stated this was an unfortunate consequence of the list context behaviour of CGI.pm. Knowing the difference between scalar/list context in perl is a classic gotcha. This behaviour lurked in code going back years - the examples you cite would all fall into the "legacy code" category. You're making the classic mistake of someone who doesn't know perl: looking at code from over a decade ago and thinking this represents modern perl. It doesn't.
A couple of other examples of critical bugs that were revealed in 2014: shellshock and heartbleed. Should we dismiss bash, C, etc, as terrible languages because of these? No. People make mistakes, don't RTFM, misunderstand language features, and bugs can exist in legacy code for years and years. Usually critical bugs.
You could have made an excellent talk from the work you did in exposing the bug(s) you found. You failed. Learn from this.
> A couple of other examples of critical bugs that were revealed in 2014: shellshock and heartbleed. Should we dismiss bash, C, etc, as terrible languages because of these?
Perl was a great language back at the 90's and early 2000 - it doesn't now.
Professional Perl programmers understood this coding error as a coding error at least in 2000, in my personal experience. If you squint, you can see it as a poorly designed interface in the CGI module (though I'm not sure how you would fix it), but your examples are passing untrusted, unvetted user input to sensitive code.
Professional programmers have understood that as bad practice for multiple decades.
Did all of those programmers and maintainers never read the tutorial for the language?
I'm certain if you went back in time and asked "What happens if a query string contains multiple parameters of the same name?" many people would look very confused, as if they'd never considered such a thing were possible. In other words, the answer to your question is "No, most web programmers neither read nor understood the documentation, because the state of web programming in those days was terrible."
From the changelog of the latest release of Perl: "CGI has been upgraded from version 3.63 to 3.65. NOTE: CGI is deprecated and may be removed from a future version of Perl."
You can still make arguments about list handling in Perl, but using a deprecated module, which has been understood by the community for years to be problematic and exists mostly for backwards compatibility, does not bolster your argument, it does the opposite.
Using large monolithic projects that were started 10+ years (Twiki: 1998, Movable Type: 2001, Bugzilla: open sourced by Mozilla in 1998) ago also doesn't lend itself well towards pointing out modern usage of Perl.
Feel free to make any argument you want, but you should use relevant examples if you hope to be taken seriously.
Edit: Typo fix: s/not be problematic/be problematic/
I love it how you picture anyone disagreeing with your conclusions,that Perl is somehow a "bad" language and everyone should just stop using it, as being (a) old and (b) think of Perl as "their" language.
You did find a real vulnerability in a common misuse of the 20 year old CGI module. What you did was good. Those were real bugs and should be fixed, and CGI.pm shouldn't have been designed that way in the first place.
But those bugs lingered for a reason. Nobody uses CGI anymore, expect for a handful of packages (of which you found three). There is no reason to stop using Perl because CGI is badly designed. There are plenty of reasons to criticise it, but at least take five minutes to get a basic grasp of the language syntax before doing so. The existance of variable contexts (which is the language feature where you can ask to flatten a list or count the elements by the calling convention alone) is simply not it.
Perl is a stable systems integration language with rock solid bindings to big database systems. The only language that even comes close is Python. The declarative object model of Moose can give you many features of a strongly typed language, and with the heavy focus on testing being ingrained in the community, it can be quite useful. That you should stay away from CGI in 2014 goes without saying.
Your logic is completely broken though. You're looking at projects written back when the only real competitor to Perl was classic ASP (PHP was still in it's infancy) and then saying that Perl as a language is terrible because you're too lazy to look at all the modern frameworks that have been added since then (have you even heard of mod_perl, Mojolicious, Dancer or Catalyst? All of them do away with CGI)
Yes, there are some bad Perl programmers, but there are bad programmers in every language. If someone missuses pointers in C/C++, you blame the developer. Or if someone cocks up the whitespacing in Python, you'd blame the developer. Of course the buck stops at the developer - he is the guy writing the code to begin with. Who else are you going to blame if someone writes bad code?
Lists aside, ironically Perl has better type safety than many other loosely typed languages due to the lack of operator overloading - thus avoiding the often overlooked triple character operators (eg ===, !==, etc). So while Perl (like any language) does have it's hidden traps, it also has protection against hidden traps that other languages exhibit. However you conveniently overlook this when ranting about how poor Perl's type system is.
The problem here is that you clearly have some deep seated prejudice against Perl and it's caused you to create a presentation that's, at best, rude, but realistically it's just down right ignorant flamebait.
If you really cared half as much about good programming practices and secure web applications as you make out, then half your presentation would have been explaining how to avoid those pitfalls you demonstrated, instead of smugly shrugging whenever the question was raised. And since you continually explained how "shit" Perl is, you could have also demonstrated the equivalent features in other languages which are safe (the problem here is that examples you'd given would have been bad practice in any programming language - as has been discussed on here already).
This is why you're receiving so many negative comments, both from HN, Youtube commenters and even half your peers in the audience that day.
Hopefully the next time you decide to give a presentation, you will put your personal feelings aside and can give a more up-to-date, balanced and better researched talk.
My main problem with the talk is not that you make fun of Perl (every langauge is easy to make fun of), but that you don't seem to have more than a superficial grasp of Perl. No one that doesn't understand a language should make recommendations as to its use.
A couple of examples:
* "You need to backslash the variable" (near-quote) - no, you need to pass the array as a reference (which can be done in several ways).
* Anyone expecting my $var = scalar @array; to return anything but the number of items in @array hasn't spent much time writing/reading Perl.
* "Most, if not all, Perl web apps use CGI.pm" - Not in a world of Mojolicious [1], Dancer [2], Catalyst [3] and others. You should check them out.
If you really wanted to send the whole list as single argument to the function, you should have sent an array reference, like this:
Perl programmers say that if you took a minute to understand the language, you should have known this. However, what they don't realize is that Perl gives you ambiguous hints about this all the time. For example, the built in push function doesn't seem to follow this logic: This function knows perfectly that the list is a single argument and the other ones are the items to push into the list. You don't have to pass a reference to the array to make it work.The push function works like this because it uses an obscure feature called function prototypes. Which is rarely mentioned in any tutorial.
When a Perl newbie sees the push function, it's more than reasonable to think that passing arrays as function arguments will work as in any other language. When they find what really happens, it can be very frustrating, specially because Perl won't give you any error whatsoever. This is one of the reasons why Perl is so difficult to learn.