I'm Brazilian and when I moved back home for a bit the country had recently inaugurated a national exam for all high-schoolers. About maybe a year after, they'd publish an anonymized data set for researchers with a nice question-by-question breakdown of how every student performed. I worked at a school at the time, and one thing we wanted to know was how WE had performed in the questions so we could gauge if there were some things our students were consistently unprepared for.
They didn't provide that data, but it turns out with a little bit of grokking and staring at that 10gb text file, you could reverse engineer it so you could extract all the kids of a given school, and aggregate all of their answers. I produced a nice little report for our admins, with the questions of the text next to how we had performed in the aggregate and state averages, as well as averages of our "competitor schools."
The best part, though, is how I remember it being a bit "bullshit" that we, a private school, could afford to do this, but since the data was actually valuable to inform practice, surely the department of education should do that for every school! Whelp, over the weekend, I computed that info for every school in the dataset, and just stored a CSV for every school in an S3 instance (this was my ridiculous caching strategy lol). Spun up a frontend where you could select your school, and nicely visually go through every question, as well as print a pdf summary of the whole thing. On Monday morning tweeted at an ed journalist, and in a few days had a spread of me in the country's top newspaper, and people emailing about jobs "at my company."
This was the most rewarding project I've ever done, and I'm sad to say nothing has come close since. It cost me $0, produced a public good that I could see was being accessed from every state in the whole country, was technically interesting, and I saw it through from start to finish over the weekend!
That you have a functional/strategic view of a particular organization/situation- honed no doubt from other experiences unique to you- has nothing to do with the view/mental model that others in and around the organization have of the organization.
More critically, your view is no more valid than anyone else's view (and in functional terms, is less valid than the mental models of your superiors), and, you have almost no power to change other people's mental models.
This isn't a unique statement about you, it is just a universal truth.
In re:
> How do you deal with superiors at your organization that appear to be without direction, strategy, awareness and action?
Once you understand that all people in an organization are independent, autonomous entities with their own mental models, and therea are no magic buttons or injections that can change those mental models, it should become clear that the only way to engage and potentially to effect anything is to build relationships with people, With relationships, you start to understand what the mental models are of the other people in the organization, what is important to them, what is not important to them, and you and they can build a shared trust.
With superiors, this practice is sometimes called "managing up." It involves setting up conversations, asking questions, learning what's important to them in the context of the organization.
It may seem like there are people who lack direction/strategy/awareness/whatever. That is a failure of one's own perceptual machinery. They may not share your direction or priorities or whatever, but they very well have their own. Mental models that lack ambition are very often the result of learned helplessness (the causes of which especially in non-profits are myriad).
When there is sufficient understanding of their mental model, and trust around shared goals, one can present to them distillations of problems that are important to them, along with solutions or win/win framings of decisions or opportunities.
Understand what success looks like to other people, and then help them achieve it.
This applies not just to superiors but to all colleagues. Understanding what is important to other people and then applying your own initiative and capability to enable their success is the secret to happiness in life.
You cannot hide anything on the internet anymore, the full IPv4 range is scanned regularly by multiple entities. If you open a port on a public IP it will get found.
If it's a obscure non-standard port it might take longer, but if it's on any of the standard ports it will get probed very quickly and included tools like shodan.io
The reason why I'm repeating this, is that not everyone knows this. People still (albeit less) put up elastic and mongodb instances with no authentication on public IP's.
The second thing which isn't well known is the Certificate Transparency logs. This is the reason why you can't (without a wildcard cert) hide any HTTPS service. When you ask Let's Encrypt (or any CA actually) to generate veryobscure.domain.tld they will send that to the Certificate Transparency logs. You can find every certificate which was minted for a domain on a tool like https://crt.sh
There are many tools like subdomain.center, https://hackertarget.com/find-dns-host-records/ comes to mind. The most impressive one I've seen, which found more much more than expected, is Detectify (which is a paid service, no affiliation), they seem to combine the passive data collection (like subdomain.center) with active brute to find even more subdomains.
When I designed Cloudflare's filters, a.k.a. Firewall Rules... we had an API written in Go, and the edge written in Lua. The risk I foresaw was that building rules that act on URLs and perform matching, and testing those, in one language (Go) that were then deployed and executed in another (Lua) would lead to a risk that the difference in the behaviour of the two engines (no matter how much they implemented the same spec) would result in a rule being created that behaved ever so differently elsewhere... for me, that was a huge hole as if it was known it would be leveraged.
The first piece of Rust in Cloudflare was introduced for this reason and for this project. We needed a single piece of code to implement the filters that could be executed from Lua, and that Go could call too. It needed to ensure that only a single piece of code was responsible for things like UPPER(), LOWER(), URL_PARSE() precisely to prevent any differences being leveraged.
> The guy from the shop who installed my new tires said his Ford Ranger tires wore out in about 6k miles. My Tacoma's tires lasted almost 50k. Sometimes car manufacturers cheap out on factory tires
TFA is about a Rivian with high-end Pirelli tires, this is not a mfr cheaping out situation.
It's clearly an alignment issue, which is not surprising at all when it's related to a deliberate ride height change.
Vehicles with independent suspension tend to have "bump steer", which varies toe with suspension travel. When the vehicle is designed to operate for sustained periods at substantially different ride heights, its suspension geometry has to adapt to these modes or it'll exhibit crap like this.
There's school of thought that pixels are point samples of a waveform (separately along horizontal and vertical axis, and that separation is useful for performance). In such interpretation resizing is downsampling that removes higher frequencies. You can then reason about this mathematically and show that you have an optimal filter, which preserves as many frequencies as possible, without bias or aliasing.
In practice, it's a bit more complicated, because pixels can form a square wave without Gibb's phenomenon. Physically they're not points. And there are all kinds of biological factors in perception.
As for reference to neural networks, I'm not sure. They're used for superresolution (upscaling), but I haven't seen them used for downscaling. They could be in theory.
If playing around with blocking bots here is one that will block some legit and friendly bots and some bad ones too. This will do nothing for headless chrome acting as a bot. Only use this on silly hobby sites, do not use in production even though most load balancers can do this and much more.
In the main site config redirect anyone not using HTTP/2.0. GoogleBot still doesnt use HTTP/2.0 so this will block Google. Bing is OK though. One could instead use variables to make this multi-condition and make exceptions for their CIDR blocks. Point "auth." DNS record to the same IP and ensure you have a cert for it or a wildcard cert.
# in main TLS site config:
# replace apex with your domain and tld with its tld.
if ($server_protocol != HTTP/2.0) { return 302 https://auth.apex.tld$request_uri; }
Then in your "auth" domain use the same config as the main site minus the redirect but then add basic authentication. Anyone not using HTTP/2.0 can still access the site if they know the right username/password. If you get a lot of bots then have an init script copy the password file into /dev/shm and reference it from there in NGinx to avoid the disk reads.
# then in the auth.apex.tld config.
# optionally give a hint replacing i_heart_bots with name_blah_pass_blah
auth_delay 2s;
location / {
auth_basic "i_heart_bots"; auth_basic_user_file /etc/nginx/.pw;
}
This will block some API command line tools, most bots good or bad, some scanning tools. Some bots will give up prior to 2 seconds so you will get a status 499 instead of 401 in the access logs. Only do this on silly hobby sites. Do not use in production. Only people wearing a T-Shirt like this one [1] may do this in production.
One may be surprised to find that most bots use old libraries that are not HTTP/2.0 enabled. When they catch up we can replace this logic using HTTP/3.0 and UDP. Beyond that we can force people to win a game of tic-tac-toe or Doom over Javascript.
There is a certain point at which the engineering is no longer what makes a company successful, it fuels an initial growth and can sustain it longer, but then the deciding thing on whether the company survives and thrives becomes the more operational aspects, the traditional sales & marketing.
With the product-market fit nailed, a solid offering out there, Twilio now occupies that later stage, and the buildboard reflects it. Everything changes at this point, because it's also not the top priority of the company to keep their own engineering happy at this point.
I work in ad tech, the difficulty in serving ads isn't showing the ads (which can be pretty annoying in itself for advanced ad formats), it's collecting enough metrics consistently enough across platforms to convince the advertisers that fraud and brand safety aren't a concern.
Verification metrics for basic ads are things like: when was the ad shown, how much of the ad was visible, which parts (pixels and video timeline), what content was also visible while the ad was shown, clicks, bot/script/adblock detection, and whether the ad was on-target (age/gender/location of user).
Ideally, these metrics are also independently verified, so Reddit's tracking and the advertiser's own provided/preferred tracking (like DoubleClick Verification).
Typically the video player or ad renderer needs to be customized to collect these metrics and understand VPAID/VAST/VMAP/MRAID for how/when to show the ad and what tracking is needed. Plus support for mixing content and ad encoding formats.
It's basically an arms race to compete against Facebook/Google's ad serving and tracking capabilities and I'd say even most 1st party premium ad publishers with full time ad tech teams often have difficulty consistently passing verification convincingly enough. So it's understandable that Reddit has very low confidence in single developer apps being able to pass the ad verification bar, much less over API which isn't a well worn ad serving path.
You made something that people got a lot of value out of. That’s incredible. The actual code and commits are just the journey. They’re not the product. I’m rather delighted every time someone shows a realistic journey full of all the things we all experience, but some of us pretend don’t happen. Thanks for being vulnerable and sharing this.
I had a professor remark once (this is circa 2009) that we should all learn a highly skilled trade around advanced manufacturing because eventually offshoring will reverse itself and by your early 30s you'll be in a prime position to profit immensely off the re-shoring of advanced manufacturing equipment and processes
Maybe he was right. I'm in my early 30s now, and I don't regret becoming a software engineer, but all the highly skilled tradesmen I've met in the last 5 years make at least what I do, often more, and have the benefit of actually owning their own businesses or real, meaningful slices of other ones.
I'll boldly admit I don't think I would like over physical labor and I do wonder about the toll of it long term, but they make a killing, especially the ones willing to travel all over the US.
Granted, the highest earner I know has both a degree in CS and Industrial Engineering, and goes all over the place working with industrial robotics and makes piles of money doing it. More than elite SV companies in the last 3 years, IIRC, if you discount the stock based compensation package (simply salary to salary comparisons) its north of 300K.
And arguably better retirement benefits, I think 12% goes to the 401K and they get bonuses every quarter right now. Well invested I bet he'd come out ahead. Can't for the life of me remember the name of the company though.
EDIT: forgot to mention the folks making tons of money on the DoD now starting to require things made in the USA and sourced raw materials from either inside the US or from friendly approved countries from the last few years either, and I don't think that's going away any time soon. They've in the last 2-3 years started paying lots and lots of money to get suppliers that are US based and source their raw material from friendly nations only, and they pay alot of money to make it worth while.
If you don’t include some measure of employee satisfaction, retention, wellness, etc, then you will optimize the system at their expense. As awful as this sounds, it’s no different than operating a piece of machinery above its capacity, or without adequate maintenance, because doing so improves some short term measure. If you can absorb the cost of doing so, then you might happily burn through lots of machines and that might be acceptable. But people aren’t machines, and you will either find yourself having retention issues, hiring issues, or like Amazon, literally running out of people willing to work for you even as they literally kill their employees.
The most important thing I take from my reading of Deming, is that there comes a point when you have to put the charts aside and deal with people. If you push people over and over and over, eventually they’re going to snap. Even if they don’t, can you live with yourself at the end of the day?
You also have to consider more factors than just P&L. If you push sales people to the point they start breaking the law, or you push machinery and operators to the point of a massive oil spill, you could jeopardize the whole business, or worse. You need to know where the limits are, and that only comes from understanding the people, and the process, beyond just the charts.
Sure! I basically use PDF Lib to generate a letter with all my evidence bundled into one pdf file. It sucks up data from my database about the user's activity. Format is the following:
Formal 1 page letter with company logo - "Dear Sir/Madam.. writing about $9.99 dispute for card XXXX on X date for user [NAME]" (all this is pulled from the Stripe API).
I then have 5 appendixes on separate pages:
Appendix 1: Users' receipt (just a pic of the Stripe receipt)
Appendix 2: User's order history (just printed, name of items, price, and timestamp)
Appendix 3: Screenshot of the user's purchases page clearly showing the item
Appendix 4: Email confirmations - I store copies of each HTML email sent, so just take a picture of this.
Appendix 5: Site activity with IP addresses and timestamps. This is all user actions on my site (viewing items, purchasing, etc.)
I created it because I had 20 disputes piling up, and I was dreading processing them all. Given that you're likely to lose the majority of disputes, it's super discouraging to put in the work, but this makes it a bit easier (I don't want to let fraudsters just get away with it!).
I worked in accessible tech for the blind for a few years. It's awesome that these guys are doing something that could be genuinely useful to a swathe of society who really needs the help, and I do wish the space got more attention and money.
However, and I hope I'm wrong here, this is looking like another case of techies with a nifty looking device that will be "available soon" but, more importantly, already tried multiple times and either never caught on for practical reasons or doesn't present a real improvement over what's already out there (0 - for three such mouth trackpads).
I wish the investors and incubators that take these companies to market would do a much better job of coaching the founders on product/market research with target users so they can focus on truly novel solutions to genuine problem spaces that are ignored, but would make a real material improvement to quality if life.
As an example, in BlindTech it was the "smart cane", which is dumb for so many reasons and will never, ever catch on, but every 6 months or so... Here pops another start up with a seed of $0.1-1M to go to market! That money would be better spent just buying "dumb" canes at a volume discount and reselling them... Or, if you insist on developing a BlindTech solution, an app that could tell a user when they missed a button on their blouse or shirt (I never worked in the paraplegic space, so I'm not sure what other challenges the users face).
In general, try to put your idea into paper using something like Lean Canvas [1], [2].
Cover the following aspects: technology, organization, people and processes and how these tie with the business context (using the lean canvas).
If your venture is going to require software development, you can
- design and develop a prototype / work on the MVP
- identify your target customer segments and talk to real people seeking their input
- build a landing page and write copy for your product
- put a list of things you need for your venture and prioritize to identify the bare-minimum needed to start
- read about company starting procedures
The key to success in a startup (apart from timing, luck, having the right idea, grit, focus, etc) is determination to see things through to completion. You have to believe in your idea and see it through to the end, where 'the end' is hopefully an exit of some sort probably years in the future. If you see yourself as "an idea guy" you will be distracted by the new shiny thing you think of, and if that happens when your startup is in a slump you'll give up before you see success.
You do need to be "an idea guy" to build a startup, but all the ideas need to be about driving the core idea forwards. You need to see yourself less as "an idea guy" and more as "a <whatever the startup is about> guy".
To that end, I'd recommend starting a side project to see if you can grow it without getting bored, giving up, etc.
I've build BlueRetro [1] an universal Bluetooth controller adapter for nearly all pre-USB gaming console.
I made a gross income of around 3K a month last year out of Royalties on the soft for each device sold.
It's Apache 2.0 software so people can do whatever they want.
I started making money when I decided to list on the GitHub README the list of manufacturers/makers that where sponsoring the project. (Only one person at that time)
Soon after the others offered to give royalty as well.
I even got a Chinese company, notorious for selling "clone" of OSHW projects, to support the SW development as well via GitHub sponsor.
I've been working on it for the last four years. I entertained the idea to make and sell the hardware myself. But in the end I learned that's it's not something I'm interested to get into. What I really like is working on the software.
It naturally pivoted into a more community driven project where multiple makers are selling various variations of the HW.
That DynamoDB VFS looks cool! I agree that the VFS api makes one think about plenty of crazy ideas. Someone is working on a VFS based on Foundation DB[0] that looks very promising. It was recently discussed here[1]
When I play around with multithreaded concurrent algorithms, I try solve a simulation of moving money between accounts in a thread safe way. It really is just deduct from one account and then add to another account.
If you don't use a mutex or a lockfree algorithm, you can end up with more money (money creation) or money destruction (money missing) when you transfer money between accounts and there is another transaction to the same account in a similar time frame. This is due to a race hazard where addition and subtraction are each individually 3 operations and if they interleave then they cause the writeback of an incorrect result and omission of another result:
This simulation has money creation, because the two transactions don't see eachother.
So I add the money up at the end of the simulation to see if it is equal to the money that the simulation started with.
When I say money sharding, I am not sharding by account. If an account has 12,000 in it, and I have 12 threads, each thread stores 1000. The fast path is a transaction goes to that thread and that thread handles the transaction if the amount of money being deducted is less than or equal to 1000. If the transaction amount is greater than what is available in one thread, then it has to be routed to other threads.
I never generate a transaction greater than what is available than all threads, so that part always works, for routing to accounts that have enough money.
An extremely slow path would have to transact a bit from multiple threads until the transaction is complete.
I've never worked on fintech or banking software, so I don't know how it works in practice but I did try implementing an order matching engine (which from my perspective is just a sort ascending + sort descending of participants bids or asks) I haven't worked on parallelising that yet.
While DuckDB is an exciting and amazing project, I think the world that will open up around it is just as exciting, and these are exactly the kinds of questions that get me excited.
DuckDB is to Snowflake/BigQuery/DataBricks/etc...
what
sqlite is to MySQL/Postgres/Oracle/etc... (let's ignore for the moment that Postgres and Oracle have HTAP modes)
In other words, I don't think DuckDB aims to replace or compete against the big OLAP products/services such as Snowflake, BigQuery, DataBricks. Instead it's a natural and complementary component in the analytical stack.
Of course you'll see in the numerous blogs about how amazing it is for data exploration, wrangling, jupyter, pandas, etc... but personally I think the questions about how it could be used in production use-cases a lot more fascinating.
Data warehouses can become quite expensive to run and operate when you either have to allow
1) front-end analytical applications to connect to them directly to do analytics on the fly, or
2) if you pre-calculate ALL the analytics (whether they're used or not) that are offloaded to a cheaper and "faster" OLTP system.
I'm excited about how DuckDB can sort of bridge these two solutions.
1) Prepare semi-pre-calculated data on your traditional data warehouse. (store in internal table or external table like iceberg, delta, etc)
2) Ingest the subsets of this data needed for different production workloads in to DuckDB for last-mile analytics and slicing/dicing.
DuckDb could either interact with your
1) push-down queries to internal tables via their database scanners (arrow across the wire. postgres_scanner, hopefully more to come), or
2) prune external tables (iceberg, delta, etc) to get the subsets (interact with catalogs) of semi-pre-calculated analytical data on demand. Think intelligently partitioned parquet files on S3.
Last-mile analytics, pagination, etc can all be done within DuckDb either directly on your browser (WASM) or on the edge with something like AWS Lambda. This could and hopefully will result in reducing the cost of keeping data warehouses around to serve up fully pre-calculated analytics to consumers as well as reducing the complexity of your analytics stack/arch.
Visual Basic controls are more like React components, I think.
ECS is, traditionally, just a way to dynamically add behaviour (components) into "blank slate" entities.
Instead of making a EvilMonster class in a game you just have it as a conceptual entity (normally it's just an integer, like a database id) associated with a bunch of components (normally just plain structs) and their parameters: Renderable, RigidBody, Collider, HasEnergy, HasTransform, AudioEmitter, etc.
You can stitch entities and components together using either code or some external data source. The "S" part of ECS is called the system, and it implements the behaviour defined by components.
The upsides of this IMO is that this is a fantastic way to structure your code. The data-oriented aspect is also great for enabling non-coders to assembly complex entities. And some other clever people (such as the sibling answer) found out this is also a great optimisation technique.
There are multiple implementations, of course, so people have different opinions.
I have grown to understand that the relational model is the answer for solving all hyper-complex problems. The Out of the Tar Pit paper was a revolution for my understanding of how to approach properly hard things:
The sacred artifact in this paper is Chapter 9: Functional Relational Programming.
Based upon inspiration in this paper, we have developed a hybrid FRP system where we map our live business state to a SQLite database (in memory) and then use queries defined by the business to determine logical outcomes or projections of state for presentation. Assuming you have all facts contained in appropriate tables, there is always some SQL query you could write to give the business what they want. An example:
> Give me a SQL rule that says the submit button is disabled if the email address or phone number are blank/null on their current order.
--Disable Order Submit Button Rule
SELECT 1 -- 1 == true, 0 == false
FROM Customer c, Order o
WHERE o.CustomerId = c.Id
AND c.IsActiveCustomer = 1
AND o.IsActiveOrder = 1
AND (IsNullOrEmpty(o.EmailAddress) OR IsNullOrEmpty(o.PhoneNumber))
I hope the advantages of this are becoming clear - You can have non-developers (ideally domain experts with some SQL background) build most of your complex software for you. No code changes are required when SQL changes.
The relational model in this context is powerful because it is something that most professionals can adopt and collaborate with over time. You don't have to be a level 40 code wizard to understand that a Customers table is very likely related to a ShoppingCarts table by way of some customer identity. If anyone starts to glaze over at your schema diagrams, just move everything into excel and hand the stakeholders some spreadsheets with example data.
The best explanation I've seen is in the book "The Secret Life of Programs" by Jonathan E. Steinhart. I'll quote that paragraph verbatim:
---
Computer programming is a two-step process:
1. Understand the universe.
2. Explain it to a three-year-old.
What does this mean? Well, you can't write computer programs to do things that you yourself don't understand. For example, you can't write a spellchecker if you don't know the rules for spelling, and you can't write a good action video game if you don't know physics. So, the first step in becoming a good computer programmer is to learn as much as you can about everything else. Solutions to problems often come from unexpected places, so don't ignore something just because it doesn't seem immediately relevant.
The second step of the process requires explaining what you know to a machine that has a very rigid view of the world, like young children do. This rigidity in children is really obvious when they're about three years old. Let's say you're trying to get out the door. You ask your child, "Where are your shoes?" The response: "There." She did answer your question. The problem is, she doesn't understand that you're really asking her to put her shoes on so that you both can go somewhere. Flexibility and the ability to make inferences are skills that children learn as they grow up. But computers are like Peter Pan: they never grow up.
I am designing an off-grid cabin with a solar panel array charging a bank of batteries with a propane generator backup. I run ethernet as power with a custom designed PCB that terminates at the outlet side where it exposes a 20 watt USB charging port and an ethernet port.
The lights are all basically cut 12v light strips inside of old light fixtures with a custom controller that also terminates PoE. The 48 volts that most PoE standards specify is more than enough to push power down the line for < 100 meter runs.
The advantage of PoE here is that anything under 50 volts is considered low voltage and does not need to follow the same rules as normal house wiring. I did not like that everything is hinging upon a beefy PoE switch so I actually made it passive PoE instead by design.
Back when I was a developer, I tended towards the competent jerk. I was pretty good, but not 10x and I was a bit short with people not a raging asshole, though looking back I am ashamed of my behavior. The good thing is that it makes it really easy for me to recognize now.
On my teams, I don’t accept incompetence, so even the nice folks get managed out quickly.
I do have a strategy for handling the 10x assholes, though. We have a difficult problem domain (lots of math and physics) so I need those smart folks. I’m also lucky to have 2 hyper-competent nice devs too. I use them like jiu jitsu gyms use their enforcers.
When an extra aggressive 20 something shows up to a gym and starts making it unsafe to train, the gym will pair them up with a black belt who will humiliate them but be super nice about it and put them in their place.
I do the same. If I see asshole behavior surfacing, I’ll pair them up with my one of my super competent devs and have a them do a in depth code review or something like that.
Most of the time it works and the bad behavior goes away. If that doesn’t work (very rarely), we’ll manage them out.
Sometimes the 10x jerk is a jerk because he’s used to dealing with incompetence. But if you can show them competent-nice behavior, they’ll often pattern match to that.
See in this case the manager has done a bad job, but the developer has also done a bad job, both in the exact same way.
The correct solution here isn't just to insist estimation is impossible and be offended that someone would ask for it - that just misunderstands how businesses work.
The correct thing is for either of those parties to say, "Okay, I understand it's not estimable at the moment. What are the steps that we need to take in order to get a reasonable estimate?"
A good PM will do this, but where there is a bad PM, a developer can step in and say the same thing. In your scenario, "The developer says “there are too many unknowns to provide any sort of estimate”." What the developer could say is, "There are too many unknowns to provide any sort of estimate - let me give you a few examples. <examples>. What I can do is come back to you after <amount of time> with a list of the unknowns and an explanation of the work required to evaluate them, so that we have enough information to estimate. Once you have that list, then you can decide if we want to proceed with the work to understand those unknowns."
I totally understand why developers get frustrated when they're asked to give estimates for things that can't reasonably be estimated with the current information available. But the flip side is it's also very reasonable for PMs to get frustrated and ignore developers who insist that estimation is an impossibility, because that's virtually never true - maybe it's impossible to do now, but there is a path to make it possible. If you work proactively to expose that path to the PM, it'll generally result in everyone being happier.
Disabling javascript breakpoints usually does the trick. Devtools detection is often done by having a `debugger;` statement somewhere and timing of it triggered
For me it's all about the access. I happily pay for Usenet access and indexers because it allows me to have a single interface for accessing all media. I don't have to play the guessing game of "which streaming service is this show / movie hosted on this month?". I just hop onto Sonarr or Radarr and it's on my NAS ready to stream in a few minutes. I still pay for Netflix and I get Disney for free from my cell phone plan so if it's outside of those I'll reach for Usenet.
They didn't provide that data, but it turns out with a little bit of grokking and staring at that 10gb text file, you could reverse engineer it so you could extract all the kids of a given school, and aggregate all of their answers. I produced a nice little report for our admins, with the questions of the text next to how we had performed in the aggregate and state averages, as well as averages of our "competitor schools."
The best part, though, is how I remember it being a bit "bullshit" that we, a private school, could afford to do this, but since the data was actually valuable to inform practice, surely the department of education should do that for every school! Whelp, over the weekend, I computed that info for every school in the dataset, and just stored a CSV for every school in an S3 instance (this was my ridiculous caching strategy lol). Spun up a frontend where you could select your school, and nicely visually go through every question, as well as print a pdf summary of the whole thing. On Monday morning tweeted at an ed journalist, and in a few days had a spread of me in the country's top newspaper, and people emailing about jobs "at my company."
This was the most rewarding project I've ever done, and I'm sad to say nothing has come close since. It cost me $0, produced a public good that I could see was being accessed from every state in the whole country, was technically interesting, and I saw it through from start to finish over the weekend!