So excited to see this coming! When we used Jupyter Notebooks to publish Practical Deep Learning for Coders, we discussed with Allen the idea of doing the same for some of his books.
Now it's actually happening. :D Even better, he's taken it further by adding cool tools such as a Jupyter-based turtle that shows inline graphics in the notebooks. I strongly suspect this will turn out to be the best way to learn Python programming when it's released.
Think Python 2e changed the trajectory of my life. I took a single Java class and hated it so much I gave up on programming. A few years later as a network engineer I had a problem that seemed like it could be scripted and ended up picking up Think Python and fell in love with the language and programming in general.
I am considering changing the trajectory of my own life, towards a more community/maker/teacher role, and I have a freelance/small business idea about teaching but I need sort of "soft syllabus" materials.
I am learning Python myself, having just never had a need for it in all of my professional web development life (I've written apps in just about every other web-focussed programming language, including Perl and Ruby).
It looks like the right language to teach general concepts in, and having a book I can draw from will help.
But I am, well, mid-life at best, and the only crisis I am experiencing is that I think much of what I have done for years is worth less than I would have hoped.
I think it is ironic that technology seems to serve people less and less (and advertisers more and more) at just exactly the same time as every possible piece of technology is falling in price and increasing in availability.
Want to make a small device with a colour LCD display, buttons, an entirely custom enclosure, a rechargeable battery and a custom PCB? The cost of doing that has never been lower. Even if you only want one.
Want to give some tool a comfortable handgrip for accessibility reasons that needs to be parametrically adjustable? The software tools are free and the prototyping tools are incredibly cheap.
Want to find the answer to a question about your local area that would have required months of library time? The software is free and shockingly complete data sets are available from governments.
So why do so many people, even adults my age and younger, feel that technology is out of their control?
I have always felt as a freelance developer that the work I am paid for is just the seed in the middle of a larger fruit, where I listen and discuss and explain and educate.
I feel I should be turning it inside out. Making the listening, discussion, education the product, and the development work ancilliary to it (because it almost always is).
What I find interesting about Python, specifically, is how many applications it is suitable for, considering its ease of access -- CAD packages, 3D printing, PCB design, GIS, microcontrollers, statistics. Children's and adult education etc.
So while I have a little time I am spending it re-orienting myself into Python as a programmer "way-of-life", and away from the tools I have been using so far (which has never, really, meant Python).
And as I do so I am getting a sense of how it might fit into me turning my freelance work inside out to become a teacher/trainer/educator/facilitator, largely for adults, to help them have better concepts of technology (even for just the humdrum business of getting suppliers to quote for bespoke projects).
A vague sketch, sorry -- but I think this might answer your question.
Just as an counterexample, I started with bash/Perl, moved to Python and really liked it and then started getting issues due to dynamic typing. Around that time, Java was chosen to be the main language for our team and once I saw the benefits of static typing, there was no looking back. Since then Python has come a long way with optional typing support and IDEs to help enforce it but by now I am forever biased in favor of statically typed languages. I still enjoy Python for writing utility scripts because it is so much more polished compared to just using bash (which has its own place obviously) - but if I have the option of choosing a language for a full fledged application, I would prefer Java/Scala etc (Rust is great but the learning curve is way too steep for most mortals).
I do admit that I haven't had the first-hand experience of optional typing in Python supported by something like PyCharm so it is possible that the tooling has brought the experience to be comparable to working with a statically typed language. If someone has used IntelliJ community edition with Python typing-enforcement turned-on, please share your experience.
It's a lot easier to appreciate Java when it isn't your first language. I think it's really important for new programmers to have a streamlined experience to their first magical moment with programming, when they understand they can do cool stuff and succeed by being persistent. ("Streamlined" may be overstating it, there will be lots of friction along the way, but it should be caused by bugs they introduced themselves and overcome by debugging, not by tangential or environmental factors.)
There's a lot of context and drudgery involved in programming that can dissuade people before they get to that magical moment, and they can blame themselves and think they weren't smart enough because they don't understand that the deck was stacked against them. For instance with Python, broken virtual environments can be really dissuading for new programmers.
I think Java's opinionated and verbose nature can be cumbersome to beginners, who have never debugged a type confusion issue and so don't give a hoot about static typing. But I think after programming in Python for a while, they'll see why eg declaring what exceptions you'll raise is tremendously helpful.
Similar for me - I had learned some Java, Matlab, C, Perl here and there but it wasn’t until Think Python 2e that I was gripped and from there read many other books and changed my career to software.
Maybe, but I think the book really was different from any book I had read before. Most books teach the language, but Think Python showed me some of the why, what each data structure might be used for, it would quickly dip into some interesting implementation details to give me a more solid foundation and then zoom out and give me perspective on why something is useful and related software writing skills and tips. It was thorough, but also concise, and the exercises were engaging, so I found myself for the first time really getting sucked into the zone with the problems and time was flying by. This hadn’t happened to me with Java or Perl.
In the abstract, yes. But I also know that Think Python is a great book.
(I came across Think Python when I was trying to help other people learn how to program. So I did not learn programming from Think Python, and Python is also not my favourite language. (It's also not my least favourite language, either. Far from it.))
Same here. I moved cities and was looking for a job... I printed a copy of this book and started going through the chapters... This book made me into a software engineer :)
Love Think Python, I have recommended it to so many learners: it balances the various concerns of a new programmer book really well. Allen Downey has a bunch of other books with somewhat similar approaches too https://greenteapress.com/wp/ -- some I do think he might have gone too far in the low-rigor side, but all the ones I've reviewed have been pretty good.
(I was sharing a table at a conference with Allen some time ago and told him how many times I'd recommended or bought people his books, and I think he thought I was bullshitting him.)
Apologies for changing the subject, but aside from real world experience (which I have and am getting at work), is there a resource of similar quality for more intermediate/advanced Python programmers? I always feel like there's a big chunk of the language or stdlib I do not know.
Whenever a new version is released, I read its What's New documentation.
Beyond that, I like to read source code, both for the stdlib and popular third-party packages. This advice generally applies when I'm learning any new language or re-familiarizing myself with one, not just Python.
I really enjoyed Fluent Python a while back as an intermediate book.
Python official docs are not completely horrible, but compared to most other popular languages (Kotlin, Scala, Rust, Go at least), the Python official docs are kind of meh.
I suppose Python docs beat C and C++ which do not have official docs besides the spec. (not counting K&R and Bjarne's books).
Also I guess Javascript does not have official docs (ie MDN is not official)
I have bought this book for every friend learning python for work purposes, really fleshes a lot out that's not taught implicitly. The data model stuff is really useful.
Humble Bundle had some nice collections on Python for many uses. For in general, I remember that Serious Python and Automate the Boring Stuff with Python were both good.
* Practices of the Python Pro (https://www.manning.com/books/practices-of-the-python-pro) — learn to design professional-level, clean, easily maintainable software at scale, includes examples for software development best practices
* Advanced Python Mastery (https://github.com/dabeaz-course/python-mastery) — exercise-driven course on Advanced Python Programming that was battle-tested several hundred times on the corporate-training circuit for more than a decade
You might want to consider books that show application of techniques in real world practical code.
For example, Effective Pandas 2 illustrates common patterns for dealing with tabular data. Along the way, it uses comprehensions, lambdas, unpacking, etc. Shows how to use pytest to refactor. Leverage visualization to understand data.
This book ALSO changed my life when I was first learning programming!! Not so much in terms of a pivot but of the way the book was written itself and how some concepts all of a sudden just made sense as a total freshman.
Now, ~10-15 years later, seeing this third edition is such a nostalgia and I can't recommend this book enough!!!
> What happened next is the cool part. Jeff Elkner, a high school teacher in Virginia, adopted my book [Think Java] and translated it into Python. He sent me a copy of his translation, and I had the unusual experience of learning Python by reading my own book.
I love these books … in my case Think Perl6 (now Think Raku) was a real eye opener. A quick scan of the new Python 3rd edition looks like it is quite dumbed down - which is probably appropriate for Python.
Cool, I didn't know about this book! I enjoyed Think Python, but this book surprises me. I'd think there are very few people learning Perl / Raku as their first language nowadays, so a book targeting more experienced programmers seems like it would make a lot more sense.
What do you mean that TPe3 is "dumbed down"? I didn't notice anything appreciably reduced in the content yet but then again I haven't gone through and compared both editions, so I was interested in your comment.
How does one do proper concurrency in Python? As in separate "processes" running with encapsulation, fault tolerance, etc. For example, how does one bundle up a TCP client in a "process" such that the process handles failed connections, broken connections, retrieving data at periodic intervals (relatively fast), receiving "messages" from other "processes", etc.?
There's Ray, Pyro, Pykka, Celery, multiprocessing, asyncio, threads, Qt, and more, but all of them have issues. And a lot of it boils down to the GIL, although "processes" that are doing I/O such as TCP and other network communication should ideally reduce the GIL effect, to my understanding (is that right?).
From what I can tell, it's basically one of the worst language choices for systems of this nature, but I am trying to figure out how to do it because I need to.
If you are I/O bound then asyncio or good old fashioned gevent will do great.
If you are CPU bound, then use multiprocessing.
If you need to accept "jobs" from elsewhere, use RabbitMQ (with or without Celery).
If you have mixed CPU/IO workload that fits the worker pattern, then you would do all 3. At the top level you have a RabbitMQ consumer, fetching jobs from a remote queue and then putting these into a multiprocessing queue processed by N=~cpucount processes. And each of these use asyncio/gevent to do their work.
I saw that at one company having RabbitMQ / Celery setup - every time a new software engineer comes in, they complain about RabbitMQ and ask why would company use it. The infrastructure was running like this without hiccups for years. At one point company has let go of many experienced engineers and this time one developer found some issue with the code and blamed it on rabbit as it was locking the queue. There were no more senior developers to contest it, so he convinced manager to swap it out for Redis. He took about two months to rewrite it. Surprise, the same issue existed on Redis. The Redis solution works fine, but has its own limitations...
So do you recommend a single Python process running asyncio or multiprocessing or both? Or do people normally split these things amongst several Python processes?
My understanding is that multiprocessing creates multiple interpreters but that it still comes across some GIL issues if all under the same Python process.
I am in general quite comfortable with the actor model, and I would ideally use Erlang/Elixir here, but I can't for various reasons.
Thank you. Do you have more information regarding this? Why do I await the socket acceptance? How do I handle the disconnect? How do I send messages into and out of the coroutine?
Can you give an example of what you consider a good solution in a different language? That will help us see exactly what you're looking for and how/whether it might be achieved in Python.
That was what I was worried you'd say. So it doesn't seem like your question is about Python at all, and all your criticism of Python seems entirely unjustified. What you're actually saying is
"I have to work in a language other than Erlang/Elixir and I don't like it"
But Erlang/Elixir are defined by the sort of concurrency model that you say you want, so as far as I can see your question would be just the same if you were asking about Java or Go.
If you have a question like "I can do this in Java or Go. How do I do it in Python?" Then people might be able to help. But please stop with the criticism of Python for not being Erlang/Elixir, since Java and Go also are not that.
This is honestly pretty frustrating trying to build a system like this in Python. If you think asyncio, coroutines, tasks, multiprocessing, threads, subprocesses, etc. are simple and understandable in Python, then I would love for someone to explain them to me in a way that is manageable.
And no, the question would not be the same. Java and Go have proper concurrency substrates. Python does not.
But RabbitMQ is just a messaging bus, isn't it? That doesn't handle the processes (operating system processes or virtual processes) actually processing the messages and those processes handling fault-tolerance for the things they're connected to.
That is correct. Honestly I’m not sure this is a good use case for Python but it’s definitely possible. I’ve used the RabbitMQ + gevent + multiprocessing pattern in the past and it works but I find the code extremely hard to reason about. If I were doing it again from scratch I’d probably choose another language with better concurrency primitives.
Rather than handling the multiprocessing and message passing yourself it would be much easier to use celery + gevent and let celery do the work of spinning up processes for executing the tasks.
It may feel limiting but my advice would be to keep all the celery job queue stuff isolated from your server, especially if you are using an async web framework. Have your web server just put all the jobs in the celery queue and let it handle executing them, regardless of whether they’re cpu or io-bound. If you try to optimize too much by doing something like leaning on celery for cpu-bound tasks but letting your web server handle the io-bound ones you’re going to be in for a world of hurt when it comes to both debugging and enforcing the order of execution. Celery has its warts but you’ll at least know where in the system your problem is and have reasonably good control over the pipeline.
Can Celery help handle state internal to the workers?
What about using Pkykka alongside Pyro such that each Pyro remote object is actually a Pykka actor? Such that Pyro allows splitting workers across separate Python OS processes and the "messaging" while Pykka handles the internal state.
Similar to how one can pass around state inside an OTP GenServer in Erlang.
In Python, it could be a class managing a session type of object, like a TCP socket connection, or managing some piece of hardware that is doing something independently of the other parts of the system, or a database writer, etc.
Oh sure, if I'm understanding you correctly. It's been a few years since I've wrenched on such code, but IIRC, you can pass any seralizable object you want to the function being executed. We used to pass around an in-memory class object, no problem. The tricky part was control flow, but that wasn't celery's problem.
This book gets overlooked in favor of other ones such as Python Crash Course but I really enjoyed reading through Think Python 2e and will read through this version as well. Check out the rest of his books on Green Tea Press.
Python Crash Course teaches Python while Think Python teachers computer science using Python. You are right that they are two different books but people recommend Python Crash Course for beginners when Think Python is a much better recommendation for them in opinion. If you just need to learn Python then PCC is a great book.
Tangential, is there a book to learn typing? I feel I need a hefty book to learn typing. Don't mind if you recommend a Typescript book or other language with similar concepts.
You mean types, right? Typing is what you do on a keyboard.
If you want to understand type systems, Types and Programming Languages (https://www.cis.upenn.edu/~bcpierce/tapl/) is the book most people start with. If that is too advanced for you, PLAI (https://www.plai.org/) is a gentle introduction to programming language theory which includes type systems.
Those books are great; though I'm not sure whether they are good for a beginner? Getting your feet wet by programming in a few reasonably typed languages might be better, before you tackle TAPL?
(By 'reasonably typed' I mean something like Haskell, OCaml, even TypeScript or Facebook's hack. But not Go, C++ or Java.)
This is an odd question to read. The first programming language I learned* was C and types were so fundamental that you HAD to learn them if you were going to do anything at all.
So now I wonder, do people really spend so much time in weakly typed languages that types seems to be a special concept? Or is there something about types that has gotten so complex that its worthy of extra study?
*I did do some BASIC as a middle schooler but it didn't stick. C did.
That is a big topic. Learning typing in Go will be easier than Typescript which is orders more sophisticated. In Typescript you can solve Sudoku in the type system! Then there is the theory of type systems. But if it is the day to day just use it, it will make sense. You are just describing what the type of something will be in code rather than let it be figured out at runtime.
Programming with types by Vlad Riscutia fits your bill exactly.
Examples in typescript (so syntax should be familiar compared to e.g OCaml) and teaches you how to model a domain in types and how to think in terms of a type system, instead of diving into the details of how to implement one.
Haskell might be a fun if indirect way to learn more about a type system. It’s a neat middle between academic typing systems and a language you can do real stuff with.
Typescript might be nice because of how it’s a gradual typing system, just try and do a bit more and more with it as you go.
I love the Jupyter-/Colab-approach and would like to join the journey... and support the project by buying the book (although I don´t need it to follow the Colab?). But without subscribing to the OReilly-learning platform it seems that buying it on amazon is the only option, right ?
It's a pre-order even on Amazon. The publication date for the book isn't until later in the year (at least for the edition available in the UK, which will probably be the same):
Green Tea Press (by Prof. Downey) has a donate button. That's probably a more efficient means of supporting the project than buying the book, considering all the middlemen.
Oh my ..., was skipping too fast through the page and didn´t see this, and also didn´t realize what he exactly is doing with his Textbook Manifesto etc.
Ok, the Paypal-Link is a no-brainer for me now. Thanks for pointing out the obvious.
Many thanks for this book! Think Python 2nd is the first book I've ever read in order to teach myself programming.
Now I've come a long way to become an actual developer shipping real products and have certainly learned a whole lot more on the topic, but I still think the first steps are the most important/difficult ones. Although I've moved away from Python towards other platforms such as Rust, the basic concepts covered in this book are still very much relevant.
I'm really excited to learn that Think Python has become more modern and interactive with this update!
I had read the second edition when I was still relatively new to Python. I even spent a few weeks trying to translate the book to Ruby. Found it a great experience, especially the friendly prose and excellent exercises.
REPL is kind of the point: did you try to type and run any of the examples in the tutorial? Just skimming it would be less useful (for the very basics).
If you already know everything in the tutorial, then try a project in Python that you did previously (familiar project, new language—Python).
This has been a great resource - I love this book! For the last 5 years, I've taught an intro to programming class at the college level and I always recommend that my students augment their resources with this book.
since the new edition will use jupyter to run all the code, i wonder if would also address how to debug efficiently in jupyter.
in the meanwhile, how has your personal experience been around this? what worked for you, especially when you use a third-party notebook solution where you may not be able to install jupyterlab extensions?
Why would anyone bother with learning anymore? When is learning enough to get started?
How will someone get started with work? What qualifies someone to start a new position?
Pretty sure only smart people will be hired for the roles.
No matter what I do, I am forever unqualified, even from junior roles.
1. Have post grad degree
2. Have internships
3. Have only part time or short tenure roles.
But can't get into any industrial role. Why?
Means, I die? What qualifies for a job?
Well one option is to go back to shithole country I came from and just stay there earning pennies. With the money I make there affording a new mac will take 2 years salary. So that is the kind of shithole I am talking about.
Now it's actually happening. :D Even better, he's taken it further by adding cool tools such as a Jupyter-based turtle that shows inline graphics in the notebooks. I strongly suspect this will turn out to be the best way to learn Python programming when it's released.
Oh and I just remembered, we even showed a proof-of-concept of converting some of the 2nd edition of this book into nbdev notebooks: https://github.com/fastai/nbdev_cards/blob/master/01_deck.ip... . That notebook is rendered as this HTML: https://fastai.github.io/nbdev_cards/deck.html