Yet another garbage article on "how to become good at X".
Idioms are indeed important, but TDD won't improve your proficiency (except in that you'll learn a testing framework and be writing more code). Neither will attempting to adhere to a "Functional" style when you've not already learned the important lessons of FP from an actually FP-oriented language. Coding guidelines? Be consistent. Discover the guidelines of the code you're working on and use that. That has nothing to do with python or proficiency.
Performance is probably the last part of proficiency that one has to learn, and arguably the most difficult. A lot of other skills have to come together before one can count "performance" as a point of their proficiency in a language, and most of those skills are really quite language-agnostic. Oh, and if you're focusing on performance but you don't yet know how to properly profile and benchmark, you're not proficient.
If you really want to become a proficient Python programmer (or any other language) try this:
1. Find the documentation
2. Learn how to navigate, search, and read the documentation
3. Find a skilled mentor or some community resource (IRC, mailing-list, website forums, etc)
4. Ask what is considered good code. Read it. Study it.
5. Create a project - little ones, like sorting your MP3 collection or mining your email for statistics.
6. Try to use the lessons learned in the code you've studied when writing your project.
7. lather, rinse, repeat. Chances are, you're not proficient until you're doing it for several hours every day - and even then, you're probably not really "proficient"... yet.
I agree with the thrust of what you're saying here, although:
> TDD won't improve your proficiency
Disagree. Knowing that you need to test your code, and going back and testing your old code gives you a real feel for how to split out code, how to write interfaces, and how to organize your objects - at least in Perl and JS, which are the languages I'm proficient in.
If you know that your routine that'll be accepting an object might be accepting a mocked up object, you're more likely to resist the temptation to get all molesty with its private methods; if you know you'll spend time trying to drive up the code coverage, you'll be more inclined to abstract and not repeat yourself.
Random plug for a book that made me a waaaay better Perl programmer: http://perlmedic.com/
re: Perl Medic... Excellent book. What I learned there has served me very well with Perl, and also helped me with work on some crufty Python as well!
Also, yeah... I should take-back the TDD comment, or at least re-word it. I just get a little knee-jerky when I hear the word TDD mindlessly spouted out as a panacea. TDD works best when combined with other best-practices, and of course, learning how to write tests that help and don't become yet another ball of mud is an art in itself!
I've learned a lot from reading Twisted and the CPython internals. The newer (2008-) code in Twisted is generally very good.
Another strategy is to skim the source code for your dependencies, without worrying about whether the code is good or not. You'll still get a good idea of the scope of the problem the library is solving, and whether it looks like the authors knew what they were doing. You'll have a crude map of the code in your code, useful when you run into bugs later.
When I was new to python I did exactly what you said, and discovered the following facts:
1. Reading a library you are using, even if it's a standard library like smtplib is fun and makes you smarter and your code better.
2. Python is such a joy to read, unlike some other languages.
3. Some libraries (I'll mention smtplib again) are such a mess, and yet reasons 1 + 2 still make it easy for you to learn from them, while also learning what not to do.
Twisted isn't very Pythonic, in that very few Python code-bases grow to have as much code as Twisted does, and the coping mechanisms Twisted has evolved to keep that code manageable wind up looking like Java because that language is also designed to make large projects manageable.
The number of classes and interfaces in Twisted is a bit of a stumbling block, but I've never come across anyone with a better, more "Pythonic" way to express the same ideas or provide the same flexibility. The ABCs introduced in modern Python are inferior to the interfaces Twisted uses, and the most recent PEP I've seen for Futures is but a pale shadow of Twisted's Deferreds.
despite mentioning algorithms (once), the links in that section are related to language-specific tweaks. so the article seems to assume that the reader is already a competent programmer (or, worse, doesn't understand that language-specific tweaks are much less important than basics).
for example, about 6 months ago i rewrote someone's C code into python. this was numerical code for processing astronomy observations and comparing them with numerical models. i reduced the running time from about 24 hours to 15 minutes. that wasn't because python is faster than C (it's not), or because i used python specific idioms (i did, but they made no real difference). it's because i replaced O(n^2) algorithms with linear ones.
Now if you re-write the C code to use O(n) instead of O(n^2) algorithms, you will reduce the runtime even further as compared to the Python runtime. Of course the time cost savings may not be as important or dramatic any more, since the Python version sounds good enough (at least for the data set in question).
Runtimes of languages still matter. Same algorithm implementation may run significantly slower in some languages than others (and this is not necessarily the language issue but interpreter/compiler issue).
The links include a pretty good explanation of functional programming. When somebody describes something as "the opposite of object-oriented", my ears perk up. I've done procedural programming since the early 80s and never quite got comfortable with OOP in any form. But I like this concept of functions taking only inputs, producing only outputs, and minimizing (or eliminating) any other side effects.
Great tips, I learned Python a while ago and have been focused on Ruby recently. I wonder what the equivalent of this post would be for becoming a proficient Ruby programmer... any advice, guys?
I'm the author of the article, so thanks a bunch for the praise! I have switched to being a professional Rails programmer four months ago and left the last 2.5 years of daily Python programming behind me. This post is more a by-product of a mail that I answered.
Maybe I'll sometimes get around and write a pendant for Ruby. To get you started - for my new job I had to read these seven books:
Nice, I've got all of those except for the CSS, Head First, and Rspec book... so I'm glad I've been headed in the right direction. Will look into those other books, I hear the RSpec book is real handy.
Thanks you so much for the tips and the list. This is why I love Hacker News.
The linked author claims that TDD is not a good approach to develop algorithms by linking to an article by Peter Norvig in which he writes tests to develop and further prove correctness of his algorithm. Maybe he didn't write the tests first, but none the less it proves the upmost importance of tests.
Btw, Peter Norvig himself has good things to say on TDD[1]. So using him in an argument against TDD whilst he's using tests is kinda self defeating.
"Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: first the developer writes a failing automated test case that defines a desired improvement or new function, then produces code to pass that test ..." [Wikipedia]
Using tests after you write your code, to test functionalty how it needs to be improved has been a common practice from the earliest days of sw dev. In other words, "the importance of tests" has never been in dispute. TDD is not so universally accepted.
Testing is beneficial, sure. But the claim that TDD ,sometimes evangelized as "not a testing method, it is a design method" (roll eyes), actually helps in "deeply understanding your problem domain" needs more validation.
Due Disclosure: I wrote the original blog post(EDIT: the blog post comparing the two Sudoku attempts not the OP for this HN discussion) . I personally think TDD is overhyped consultantware snake oil, but that is just my personal opinion. If it works for you, more power to you.
Yep, and witness Rake releasing an update that breaks Rails, Rubygems releasing an update that breaks Rails, Rails 3 being a performance nightmare, gem authors nuking each others' namespaces...
I think in general that Ruby has one of the better communities out there (I'm a rubyist myself) but all this noise about TDD and cute DSLs (RSpec, Cucumber...) hasn't made a material difference in Ruby code quality compared to any other language.
You did write that blog post? Nice job, man! I skimmed over the rest of the blog and it's got some pretty nice articles on. I especially like the one concerning VIM and Python[1].
It's just a question of planning ahead: for instance I find the best way to code in Python is to lay out all the whitespace first, then add the text in at the end.
Idioms are indeed important, but TDD won't improve your proficiency (except in that you'll learn a testing framework and be writing more code). Neither will attempting to adhere to a "Functional" style when you've not already learned the important lessons of FP from an actually FP-oriented language. Coding guidelines? Be consistent. Discover the guidelines of the code you're working on and use that. That has nothing to do with python or proficiency.
Performance is probably the last part of proficiency that one has to learn, and arguably the most difficult. A lot of other skills have to come together before one can count "performance" as a point of their proficiency in a language, and most of those skills are really quite language-agnostic. Oh, and if you're focusing on performance but you don't yet know how to properly profile and benchmark, you're not proficient.
If you really want to become a proficient Python programmer (or any other language) try this:
1. Find the documentation
2. Learn how to navigate, search, and read the documentation
3. Find a skilled mentor or some community resource (IRC, mailing-list, website forums, etc)
4. Ask what is considered good code. Read it. Study it.
5. Create a project - little ones, like sorting your MP3 collection or mining your email for statistics.
6. Try to use the lessons learned in the code you've studied when writing your project.
7. lather, rinse, repeat. Chances are, you're not proficient until you're doing it for several hours every day - and even then, you're probably not really "proficient"... yet.
/rant off