I got until Unit testing the verbose name for pub_date that's not needed, nor you should unit test that, what are you testing? That functionality or 'unit' you didn't write and I bet is a test in Django.
And then it goes on and on about testing things like 'relationships'.
I've been running away from setUp and TearDown because they ruin test readability (maybe I'd just stick in the selenium browser).
Also most of the unit tests written there are not running in isolation which means... they're not unit tests! Those are integration tests.
If you base your tests in the techniques described there you are going to end up with a very slow test suite and very fragile tests (meaning stuff breaks where it shouldn't break).
I like to run:
nose: to run tests as functions not classes
pinocchio: to see specs in color
nose-quickunit: to find changes in my repo and figure out what test to run (when not TDDing)
coverage: stats
mock: mocking library
Hi there, author here, and thanks - that's all very valid feedback.
I totally accept the point on the first few "unit" tests being reliant on the database. I took the decision that testing behaviour, and thus introducing the Django ORM early on, would be better in terms of what the students are learning... I think that a discussion of how you could re-write those tests without using the database, perhaps in a later tutorial, would definitely be a good idea, and it could then take in some of the pros and cons of making things maximally granular...
I spend a bit of time later on discussing "testing philosophy", ie what to test and what not to test - I'd be interested in your feedback on that?
Sorry if I was rude :-). I think your example is good enough if you don't have a starting point (it happens a lot on Python) though, hope I didn't sound like I hated the article.
I think you shouldn't test what is already tested, since, if a verbose_name fails it's likely not your fault, but still the verbose_name would fail, I'd not test attributes unless they call a function of yours, .e.g.: Generating a choices tuple of tuples from a function (or classmethod) you wrote.
A friend of mine wrote this: https://github.com/gabrielfalcao/unclebob is a good learning point IMHO, but I'd recommend just reading what he did and making your own.
About the unit tests, I think they should be isolated, I have this rule, I just test the methods of a model, and don't even instantiate the model (database access could be used in __init__). At the beginning this could be hard to do, but, to me it nows comes natural since mocking 'self' is fairly easy, now, I don't unit test legacy code, so I wouldn't recommend this approach if you're not doing TDD. To deal with legacy code, I'd write the functional tests just like you did, and rewrite things with TDD (writing the unit test first).
The area where I have most issues is integration tests, it's hard to come with a convention on how and when to write them. Again, I know what an integration test is (most of the unit tests you described are integration tests), I just don't know when to use them, or when they're needed if you have functional tests or unit tests.
Your tests, or the output they produce, look awesome. Can you elaborate on your setup? Just today I was looking for an up to date introduction for testing in django, but could only find some older blog posts. So I would be glad if you could detail your setup a bit.
I've basically been following some Ruby people for the last couple of months - and worked at a Ruby shop myself last year - they happen to know a lot more about testing than us, pythonistas, but hey, it's very good to learn, at least Mock being included into Python3 is a step in the right (well that depends what camp are you in) direction.
The first one looks like nose with coverage, and the second like pinocchio with coverage. There are various ways to get those running with Django, but yes, a blog post would be nice.
Just don't get all caught up in 100% test coverage, it's not very useful. You should test edge cases that run the same code more than obvious cases that run new code.
It's a fantastic tutorial and I hope everyone will start testing their code at least with selenium. But.. it's still not unit-testing (and not TDD) at all. And in things like this, when you break something at the bottom, tons and tons of tests will break.
In unit-testing tests are glanular, units are small, and you build a pyramid small step after small step.
Because TDD is about test-driven design. And functional tests don't do any impact on design.
TDD is about creating code in a way, suitable for unit-testing and creating code step-by-step, keeping each step small and all branches tested.
TDD does these restrictions:
1. don't write line of code until you wrote failing test.
2. don't write line of test until you wrote code that "fixes" failing test.
So, by these restrictions, you will always try to cover all possible branches of your code, all possible different situations and corner-cases. While with functional tests, it's very hard to do that.
I don't think I accept your critique then - I make a clear distinction between "functional" tests (ie, the selenium ones), and "unit" tests, and I write both types
Although you may have a valid point that my "unit" tests aren't unit-ey enough to be called real "unit" tests, because they rely on django and the database - but I still think they serve the correct purpose.
Whatever you call them, they very clearly are written first, before any lines of code, and each line of code is only written in order to fix a failing test.
Maybe you're saying that if my "unit" tests aren't "unit-ey" enough, becauye they are too reliant on Django and the database, they don't work so well in the sense of test-driven design? If so, how?
Let's look at a specific example, ie how to design a model for Polls.
my approach, in pseudocode:
instantiate a model
assign some attributes to it
save it
retrieve it from the database
assert the retrieved object has the attributes assigned earlier, with the correct values & types (including a datetime value)
a more "unit" approach:
instantiate an object
assert it is a subclass of django's model types
assert it has default attributes with correct default types
finally, use some django voodoo or metaclass inspection to confirm that the datetime field, whose default value is "None", is actually a working datetime field...
That last bit is the clincher - in this case, I think particularly due to the datetime field, using a more "unitey" approach actually makes it /harder/ to use TDD to design the object... Whereas using my approach makes it very natural - the test influences the design, because the test uses the object in the way that the code will, the test is a client of the final objects' API, in a more complete way than a "true" unit test would be...
So, I think in this specific case, making tests "more unit-ey" wouldn't help make for better test-driven-design.
I do accept the point that if you're tests aren't isolated from one another, then it's very hard to refactor one area of code without breaking lots of tests in other areas. And I think that where it's my own code, I'm very prepared to spend effort isolating unit tests from each other, using Mocks or whatever else. But when it's Django code, which has a stable API, I'm not that worried by exercising it in unit tests...
And then it goes on and on about testing things like 'relationships'.
I've been running away from setUp and TearDown because they ruin test readability (maybe I'd just stick in the selenium browser).
Also most of the unit tests written there are not running in isolation which means... they're not unit tests! Those are integration tests.
If you base your tests in the techniques described there you are going to end up with a very slow test suite and very fragile tests (meaning stuff breaks where it shouldn't break).
I like to run:
nose: to run tests as functions not classes pinocchio: to see specs in color nose-quickunit: to find changes in my repo and figure out what test to run (when not TDDing) coverage: stats mock: mocking library
And my tests look like this:
http://cl.ly/1R2T0G282h1z331C1A2Y
or like this:
http://cl.ly/20263T1y2s402B1v2D3R
I think they're very pretty and run fast :-)