Excellent post, thanks for sharing! A couple of comments from a person not in django camp.
First of all, if we're talking about apis I wouldn't go with django in a first place. Of course, familiarity matters, but django requires a person to pick up quite a lot of concepts (forms, models, serializers, views etc) together with rather confusing multi app structure, which takes time to get used to and it's clearly visible from this post given the amount of things to be aware of.
Second, the rule on types clearly highlights how much pain does it bring to do a big project in a dynamic language. I mean, just stop for a second and think about it - you need to do special tricks to help yourself to understand even whether a variable is a list or a scalar value and I'm not even talking about mutable classes where you can't be sure about which fields are legit and which are accidental / monkey patched.
After doing a smaller project in django I've decided to try to do it in go only to be amazed how much faster it was, precisely because of the absence of problems mentioned above. It's not much harder to code (on the same level actually), type system doesn't stand in the way all the time, but at the same time you're in total control of your code.
There are few more points I would like to add:
- control inputs and outputs of your api. Sometimes I see people just passing an object with input data down the stack even without enumerating all the fields, because of reasons. Aside for the obvious security risk it also brings a lot of uncertainty in what the downstream code expects. It's the same antipattern as with javascript where functions are often defined as `function abc(params)` where params - is just a hash of unknown nature. Why is it bad? Because it's impossible to say what the code expects to find in this object without reading the full code. Same goes for the output. Sometimes people simply dump a model to the json and assume that's fine. Here there is the reverse trouble - maybe you know what's being sent but you have no idea, whether it's being used or not and can't deprecate/remove fields without checking all the client code.
- Sometimes you can write the code in a way that prevents you from messing it up. E.g. sqlboiler in go land. It generates the model against the database instance, which means that it's not possible to add business logic to them and that limitation makes it completely safe to work with them from anywhere in the code without a fear that there are some side effects there
- Be suspicious to REST paradigm. It sounds nice in theory like many other things do, but in fact it's really limiting, especially when you're doing an api for an spa. It'll very soon need an endpoint that won't map into any resource or should be a combination of several of them and by strictly sticking to REST you can soon find it really hard to implement even innocent demands from the frontend.
- Use minimal amount of dependencies. This is particularly painful in node land where you get 300mb of node modules just to get started. Every new dependency is something that can potentially break and/or bring security risks and/or get abandoned. If some functionality can be written in a couple of hours it's worth just doing it rather than depend on a random thing from github.
- Write your decisions as comments in the code. The code should be self descriptive in what it does but it won't tell why it does that and in many cases that's the most important thing to know.
- Be aware of ssr and spa combo. It's pretty popular nowadays and for a reason, but it also brings a load of complexity, since ssr is done with libraries like react which means that in addition to your django application you'll have to run a node app somewhere and think about syncing state back and forth or duplicate data access layer. The same goes for spas - it's worth thinking hard about whether there is a real need for it, since it immediately brings more complexity
- Maybe I've misread it but I strongly disagree on saving sanitised input in the database. Any modern orm will make sure that you won't get an sql injection, most of templating systems either escape the data by default or can be tweaked to do so. In return you get the flexibility of adapting the output as you need it in a specific case. Also, just think about the case when you sanitize user input only to realize months later that you need to do it differently. What will you do in this case?
- Something not mentioned there - I think as a general rule developers should try hard to avoid getting more external dependencies like queues, storages and so on. For a really long time a postgres instance can easily cover all needs and is super robust. With every new external dependency things will quickly get more and more complicated
- Structure your dployments to make it easy to spin up new services / cronjobs on the same code base. Complexity lies in centralization. If there is one huge app to do it all that's being deployed as a single unit, it'll very quickly become quite scary to deploy it. If it's possible to separate individual chunks of work to run independently it's almost always beneficial to do so. If the code independent and can be deployed in separate units, it's million times safer to develop and deploy compared to one monolithic superservice.
Found these points to be very well thought out especially as it relates to the impact of type safety upstream and downstream and minimizing extensive dependency hierarchies.
Can you point me to some resources on using go to design services, best practices, etc? Are there other approaches that are possibly better than go which you are considering? What would you suggest as an alternative to REST api?
First of all, if we're talking about apis I wouldn't go with django in a first place. Of course, familiarity matters, but django requires a person to pick up quite a lot of concepts (forms, models, serializers, views etc) together with rather confusing multi app structure, which takes time to get used to and it's clearly visible from this post given the amount of things to be aware of.
Second, the rule on types clearly highlights how much pain does it bring to do a big project in a dynamic language. I mean, just stop for a second and think about it - you need to do special tricks to help yourself to understand even whether a variable is a list or a scalar value and I'm not even talking about mutable classes where you can't be sure about which fields are legit and which are accidental / monkey patched.
After doing a smaller project in django I've decided to try to do it in go only to be amazed how much faster it was, precisely because of the absence of problems mentioned above. It's not much harder to code (on the same level actually), type system doesn't stand in the way all the time, but at the same time you're in total control of your code.
There are few more points I would like to add:
- control inputs and outputs of your api. Sometimes I see people just passing an object with input data down the stack even without enumerating all the fields, because of reasons. Aside for the obvious security risk it also brings a lot of uncertainty in what the downstream code expects. It's the same antipattern as with javascript where functions are often defined as `function abc(params)` where params - is just a hash of unknown nature. Why is it bad? Because it's impossible to say what the code expects to find in this object without reading the full code. Same goes for the output. Sometimes people simply dump a model to the json and assume that's fine. Here there is the reverse trouble - maybe you know what's being sent but you have no idea, whether it's being used or not and can't deprecate/remove fields without checking all the client code.
- Sometimes you can write the code in a way that prevents you from messing it up. E.g. sqlboiler in go land. It generates the model against the database instance, which means that it's not possible to add business logic to them and that limitation makes it completely safe to work with them from anywhere in the code without a fear that there are some side effects there
- Be suspicious to REST paradigm. It sounds nice in theory like many other things do, but in fact it's really limiting, especially when you're doing an api for an spa. It'll very soon need an endpoint that won't map into any resource or should be a combination of several of them and by strictly sticking to REST you can soon find it really hard to implement even innocent demands from the frontend.
- Use minimal amount of dependencies. This is particularly painful in node land where you get 300mb of node modules just to get started. Every new dependency is something that can potentially break and/or bring security risks and/or get abandoned. If some functionality can be written in a couple of hours it's worth just doing it rather than depend on a random thing from github.
- Write your decisions as comments in the code. The code should be self descriptive in what it does but it won't tell why it does that and in many cases that's the most important thing to know.
- Be aware of ssr and spa combo. It's pretty popular nowadays and for a reason, but it also brings a load of complexity, since ssr is done with libraries like react which means that in addition to your django application you'll have to run a node app somewhere and think about syncing state back and forth or duplicate data access layer. The same goes for spas - it's worth thinking hard about whether there is a real need for it, since it immediately brings more complexity
- Maybe I've misread it but I strongly disagree on saving sanitised input in the database. Any modern orm will make sure that you won't get an sql injection, most of templating systems either escape the data by default or can be tweaked to do so. In return you get the flexibility of adapting the output as you need it in a specific case. Also, just think about the case when you sanitize user input only to realize months later that you need to do it differently. What will you do in this case?
- Something not mentioned there - I think as a general rule developers should try hard to avoid getting more external dependencies like queues, storages and so on. For a really long time a postgres instance can easily cover all needs and is super robust. With every new external dependency things will quickly get more and more complicated
- Structure your dployments to make it easy to spin up new services / cronjobs on the same code base. Complexity lies in centralization. If there is one huge app to do it all that's being deployed as a single unit, it'll very quickly become quite scary to deploy it. If it's possible to separate individual chunks of work to run independently it's almost always beneficial to do so. If the code independent and can be deployed in separate units, it's million times safer to develop and deploy compared to one monolithic superservice.