Hacker News new | past | comments | ask | show | jobs | submit login

Thanks.

I've done this with iteration 2 (current version) or my backend but now I'm not a one man shop and have a team (well one other junior backend and a senior mobile). And we are architecting for some pretty major new features so it's interesting trying to strike a balance with both what I've learned along the way and what it seems 'best practice' is when it comes to python




Careful with where you read "best practice" from. Some folks like to brag about their new high-scalability architecture, but it's way over-engineered for most software.

The pattern I described of using SQLite tables to mimic a message queue will be very easy to switch to PostgreSQL or MySQL. If you hit serious scale you can switch to something like Kafka or Kinesis. That'll be a bit more involved, but hopefully your revenue and team scaled to match the compute needs. When you switch to Kafka, you can use Spark or Storm instead of cron. If you switch to Kinesis, you might choose AWS Lambda. So you're not writing yourself into a corner.


For an instant latency boost, you can also pop the sqlite db directory on to a memory tmpfs.


I've always wondered about this: doesn't the OS read buffer pretty much keep the whole thing in memory anyway, if you have memory to spare? Does it make that much of a difference?


It's not the reads but the writes that'll get you. Reads are easy to scale.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: