It's worth imagining if curl was invented today...
How large would the team be behind it? How many stars on Github? How many open pull-requests?
How big would the patreon be? The Github sponsors? What Fortune100 would sponsor it?
The open issues count? The build steps? The installer packages? The docker image? The docker-compose? The rust rewrite?
Wiki pages? The reactjs website? The nextjs rewrite? The CSS framework that would be born from the rewrite?
The incomplete documentation, because of the Discord? How big would the Discord be? What about the subreddit born to cry about the lack of documentation because of the Discord?
What about the startup that would offer you curl as a service? Curl as a lambda? Curl one click install in CPanel?
How many milliseconds would curl take to just start up? How much RAM would it take?
How many developer resume would claim to be proficient at curl?
Somethings are amazing because they were made at a time where things were simply made... Not taking a stab at anyone or anything in particular, just reminiscing and imagining...
I'm a business consultant and surprised how Daniel missed a big opportunity here. He should start selling to big enterprises by making curl a subscription-based model. Then he could hire thousands of developers to support him so he never has to do the grunt work himself and completely focus on monetization strategy and a rewrite in nextjs. Then he could start an ad-based model in partnership with Facebook for free use to support the free tier. Daniel, if you're reading this contact me. We can make a good deal.
HTTP is in the base, but HTTPS is on a subscription model with payment based on volume, cipher, and number of hosts communicated with. Hosts can pay to make their endpoints cheaper to talk to, but curl adds sponsored fields to all JSON sent to endpoints unless a "clean JSON" subscription is purchased. curl of course sends telemetry to its developers' endpoint, at the usual rate, and refuses to work if that endpoint sends a 5xx or 4xx response. Better hope Cloudflare doesn't have one of its hiccups while you're debugging a RESTful endpoint!
(Telemetry may be used to ensure curl isn't employed in violating the law, trade secrets, business model, or terms of service of any business partners. All of this is strictly necessary because the curl developers have no legal or financial presence in the EU or California. Your continued use of curl constitutes your assent to these practices.)
I am hoping this is tongue in cheek, but also a little scared it might be in earnest... on the plus side trying to imagine what ad-supported curl calls might look like does at least make me giggle a little.
Would love to hear the explanation for that. When people use postman I'm wondering, why not use curl itself? Never looked into postman and know well enough even in C, a simple GET (or POST!) request is a few lines of code.
It’s a convenient way to store and sort lots of curl requests for later use, its best cases are when you’ve got libraries of APIs you interact with semi-regularly (but not in such a consistent way that you’d be better off just writing a script). The paid version exists primarily so that you can have such a collection synced between an entire team automatically…but I’ve never worked somewhere that would be worth the cost.
For one off requests, yeah there’s no point to something like that.
It's great for introducing people to HTTP, I always refer inexperienced new-hires to use it since it has proven to very effective at getting them up to speed. In many ways you could say Postman offers a GUI to the protocol itself, making it something you can visually explore.
I see stuff like curl as a more advanced tool for people who already know what they are doing. In fact Postman even offers a curl "export" option that lets you see how to do whatever you have done in Postman, using curl instead. So arguably, it's also a good introduction to curl as well.
Its user interface is kinda meh but it's pretty good for documenting and demoing an API across a developer team. However, Postman owns the data and upsells you a cloud solution.
A set of curl scripts or a .http file (httpie) held in your project's git repository is IMO preferable, most editors allow you to run these directly from the editor.
You could also try Hurl [1] which is a cli tool using plain text and libcurl for run and test HTTP requests. I'm one of the maintainer and very grateful to curl/libcurl! My project couldn't exist without it.
The simple way to do this is just to declare that a new curl is needed in Rust, and you'll get thousands of people lined up to work on your "breakthrough" project. Then get seed funding for this incredible task and you'll have a business!
So true! All of this makes me feel like I missed out on the golden era of open source. I mean I was there 25 years ago but my programming skills were not good enough to write something like ls command, let alone something like curl!
If I knew as much about programming 25 years ago as I know now, maybe I may have thought of writing something like curl or wget too and make a successful project out of it. But alas! Those days are gone. And developing a new project these days comes with so much baggage. Developing something like curl from scratch today would be a mammoth task. The HTTP/HTTPS protocol spec has become huge! And so many compatibility issues to consider and resolve even for creating an MVP that can get some minimal adoption.
What good problems are there today that curious developers like me can work on? Problems that do not have a huge historical baggage and compatibility issues to worry about? Problems that are still new and tractable enough for a single person to work on and get an MVP out?
It doesn't have to be anything new; you could contribute to the SDKs of some of the current-day new programming languages like Rust and Go, that will have reimplementations of what curl does as well.
Nothing I can think of that will make you important though.
You could tackle curl as a service. I don't think anyone has done that one yet, Yahoo pipes got close, but we don't talk about Yahoo without losing all credibility.
What was the most depressing funny post I’ve seen. Ever. Great job. I genuinely feel sad and nostalgic now. I know there’s a lot of love for Discord but oh boy, where do I start…
I wasn't sure what "driven" meant despite hearing it all the time, so I looked in the dictionary:
"Someone who is driven is so determined to achieve something or be successful that all of their behavior is directed toward this aim"
That fits with this early part of the article:
"The concept behind curl development has always been to gradually and iteratively improve all aspects of it. Keep behavior, but enhance the code, add test cases, improve the documentation. Over and over, year after year. It never stops. As the timeline below helps showing.
Similarly, there was no sudden specific moment when suddenly curl became popular and the number of users skyrocketed. Instead, the number of users and the popularity of the tool and library has gradually and continuously grown. In 1998 there were few users. By 2010 there were hundreds of millions."
I love curl. It's become one of my favorite pieces of software partly due to the fact that I use it so often but mostly because when I do run into problems, the documentation is so good. Thanks for making useful stuff!
Open the network tab in your browser's dev-tools. Right click on any interesting request and "Copy as cURL." Now you're well on your way to using cURL to do something interesting.
I would also say both has it's place. Wget is the downloader, curl is for when I really need control over the request, headers, body and so on (so interactions with APIs, testing, ...)
Indeed. I really enjoyed the article, and I am grateful for the tool as well. My favorite section was "Staying relevant" which explains (IMO) how it is possible for this project to have not only kept such staying power but exploded in use.
Also the "2019" section made me lol:
> I started working for wolfSSL, doing curl full-time. It just took 21 years to make curl my job.
Finally: thank you to Daniel Stenberg, Rafael Sagula, and the many, many contributors for making this tool.
It's not so bad for most people. If you're in one of the countries the US likes, you can apply for an ESTA which lets you stay in the country for up to 3 months or something; way longer than you need for a conference. It was very painless when I travelled to the US from Norway for a holiday some years back.
Of course, it's not as easy if you're not from one of the 40 countries eligible for the Visa Waiver Program. But from what I can see, Daniel is in and from Sweden, which is one of those countries.
> when I was refused to board the flight due to unspecified "problems with my ESTA". As my employer at the time, Mozilla engaged some people on both the American as well as the Swedish side to try to figure out what was wrong and what we could do to correct the situation. Unfortunately, no one would offer any clues or information about why I was denied
It's apparently not that easy when the US decides it's not enough.
There are a ton more people for whom it is very bad. ESTA only includes maybe 600-700 million people. There are probably 7 billion people for which it is hard to very hard to get any US visa.
Sorry, you're absolutely correct. What I meant to express is, for a lot of people, particularly likely English-speaking programming language conference attendees, it's not that bad. My parent comment sort of made it sound like the only way for anyone to get to the US is through a years-long VISA application process.
But I should not have said "most people". And now knowing the context of Daniel's situation, my comment was probably unnecessary in general.
> My parent comment sort of made it sound like the only way for anyone to get to the US is through a years-long VISA application process
That was not really my intent.
My comment was quite snarky, this was the intent. I got a visa myself quite easily, but a lot of people actually argue for not organizing conferences there because of visa issues.
A former colleague of mine wanted to present a paper and got his visa issued after the conference, most likely because his passport is a Lebanese one. He's been living in France for years. His PhD supervisor had to present for him instead. This colleague wont best paper awards but no, he is not going to present them in the US.
My (more) serious argument is yes, it is this bad, and good enough for a lot of people is not satisfactory. We need places that don't arbitrarily exclude people so much. The US needs to fix this visa issue and until then, it would be best if events were organized elsewhere.
Not all interesting people doing research or involved in programming are American or EU citizen. This happens everywhere, including from countries if you happen to leave have set foot in or lived, you are practically banned from going to the US.
If I ever go to Iran, I can say goodbye to ESTA for the rest of my life. Even if it's because of a plane issue and the plane needs to do an emergency landing.
Goes without saying: Please do not apply this frustration to American citizens. Most american citizens don't realize or have experienced how difficult the government has made traveling to the US. It is incredibly frustrating as a native citizen to be interogated for, having the right, to return home. It's insane about how aggressive they have become for selecting out and collecting biometrics for visitors.
HTTP client libraries on the various languages still seems so lousy with excessive boilerplate and all sorts of complexity to handle cases that are beyond rare.
> with almost 150 CVEs since its inception. Just now for the 25nd aniversary half a dozen were added to the list.
These ones?
141. We are not aware of any exploit of this flaw. Severity: Low
140. We are not aware of any exploit of this flaw. Severity: Low
139. We are not aware of any exploit of this flaw. Severity: Low
138. We are not aware of any exploit of this flaw. Severity: Medium
137. We are not aware of any exploit of this flaw. Severity: Low
...
> The dependency list is an absolute security nightmare:
-- Curl claims that it supports so many protocols but I do not see many examples where they are in action, for instance, how to use ssh protocol to perform transfer between two machines? Where can I find such examples?
-- Are there any examples of a fully functional high-level data motion tool that is based on Curl?
-- Has anyone used Curl to perform data transfers at large scale such as terabytes of data? I would love to read more about it if there are any pointers describing such transfers.
> for instance, how to use ssh protocol to perform transfer between two machines? Where can I find such examples?
That's out of scope for cURL, use `scp` for that.
> Are there any examples of a fully functional high-level data motion tool that is based on Curl?
What's data motion?
> Has anyone used Curl to perform data transfers at large scale such as terabytes of data? I would love to read more about it if there are any pointers describing such transfers.
There's a few posts on Stack Overflow about that, but they go up to gigabytes. cURL is not the best solution for transfering terabytes of data. http://moo.nac.uci.edu/~hjm/HOWTO_move_data.html seems to be a good overview of the challenges (and tools) to use to securely and reliably transfer terabytes of data, referencing rsync, bbcp, bittorrent, etc.
> Curl claims that it supports so many protocols but I do not see many examples where they are in action, for instance, how to use ssh protocol to perform transfer between two machines? Where can I find such examples?
SSH file transfers use either SCP:// or SFTP:// urls, not SSH:// urls.
> Curl claims that it supports so many protocols but I do not see many examples where they are in action, for instance, how to use ssh protocol to perform transfer between two machines? Where can I find such examples?
You could try it yourself! The man page is pretty good.
How large would the team be behind it? How many stars on Github? How many open pull-requests?
How big would the patreon be? The Github sponsors? What Fortune100 would sponsor it?
The open issues count? The build steps? The installer packages? The docker image? The docker-compose? The rust rewrite?
Wiki pages? The reactjs website? The nextjs rewrite? The CSS framework that would be born from the rewrite?
The incomplete documentation, because of the Discord? How big would the Discord be? What about the subreddit born to cry about the lack of documentation because of the Discord?
What about the startup that would offer you curl as a service? Curl as a lambda? Curl one click install in CPanel?
How many milliseconds would curl take to just start up? How much RAM would it take?
How many developer resume would claim to be proficient at curl?
Somethings are amazing because they were made at a time where things were simply made... Not taking a stab at anyone or anything in particular, just reminiscing and imagining...