Hacker News new | past | comments | ask | show | jobs | submit | ctrlaltdylan's comments login

Is the Google Sheets API rate limit open enough for actual production use?

I thought it was pretty restrictive, no more than 60 writes per minute, but I'm not sure about the reads restrictions.


I used Google Sheets as a data source that business people could update, but eventually we moved away from it as we found it unreliable. We would get an occasional error (maybe a 429) even though we were polling the sheet once a minute (we had a few other sheets that polled once every few minutes).

This manifested as an issue when doing a deploy but being unable to get critical data. We added retries and stuff like that but it seemed not great to run a business of something that isn’t designed for this purpose.


Perhaps the dreaded 503 Internal Error ?

I'm convinced most of the people in this thread haven't tried working much with Google Sheets API at scale. Most of the time it's fine, then it will have days where 30-40% of the calls (as measured by Google Cloud console API monitoring) will throw an internal error which Google advises the option for is to "try again later". Also API calls that take up to 4 minutes (?!) to return (again as measured by their own API monitoring tools in Cloud console).

It's too bad because I otherwise really like this approach.


Yes. I used Google sheets as a database to build a website and ran into this issue. The worse part is, if you come across the limit there’s not much you can do but wait or rate limit.

Another problem I had is an API change one year in.

I would not use Google Sheets again. Maybe I’d try Airtable, Notion, or some other similar platform where the API access is more of a priority to the company.


For reading sheets, it's better to use the "share as CSV" option since that gets cached pretty well w/o limits


I've resisted this temptation to integrate with google apis for these 2 specific reasons rate limits and api changes.


For now I`m setting no restrictions. Since it is an MVP, I`m trying to understand what a basic and a hard user would be. After a while, Ill figure out how to charge for it and what limitations should a free and a paid user have.

My Google API rate limit is way bigger then 60/minute.


Couldn't you cache the reads? Not many usages really require real-time from their data store.


Do you really want to deal with caching logic for what should be a simple API call? Sounds like a convincing argument to use whatever this product is.


It's max a couple of hours work to cache in some local database like sqlite or in memory.


client = (APIcall) => redis.get(sha1(APIcall)) || { res = api(APIcall); redis.set(sha1(APIcall),res) return res }

Not that hard. Like 10 lines of code to get a decentish cache going.


Assuming you have Redis


Redis uses like 5mb of baseline RAM and can be deployed in a few lines of docker-compose.


I'm not allowed to do that where I work. License is a no-no, can't run jobs without red tape, and there's no Docker either.


Ok? I'm surprised your work lets you build a whole product ontop of google sheets, then. Also, why did you delete your original comment on not having a server?


I deleted it cause I realized this thing has a server (probably). Was mixing it up with other people's projects that didn't have one.

They're internal tools, but big ones. And I'm surprised too. You won't hit too much resistance doing things the well-supported ways, but for some reason there's no well-supported way to run a cache.


Hell, just stick the data in memory.


Valid strategy


Also this opens the possibility of running these models on Node.js serverless functions no?

That certainly also has to open up possibilities for on-demand predictions?


This is interesting. Would love to see some examples of this.


What a throwback.

This was the only distro that I could fit on a memory stick, which were novel at the time (and memory was $$$ if you can believe that).

I stuck this into a machine I made from parts I found at our recycling center, and threw them in a shoebox.

Good times.


We provide ID verification specifically for eCommerce to help prevent chargeback fraud: https://getverdict.com

This is the first where I've heard of using IDv for preventing triangle fraud on the fulfillment side.

Just curious - how does this fraud harm you the delivery service? The chargeback hits the merchant only no? Or are you the merchant in this transaction as well?


Similar to you, chargeback is a concern (triangle fraudsters using stolen credit cards and all). We're not the merchant, normally that's a brick and mortar retailer. In addition to chargeback Generally, I really hate the idea of seedy users exploiting the service and feel obligated to root that out.


All of these distortions as well as a practical exercise identify and help correct them are available in this book called "Feeling Good" that I think was published in the 80's or 90's.

I've seen them recycled targeting different audiences but the same core principles are used. I just recommend "Feeling Good" because it's on Libby / Amazon < $5.

I started keeping a weekly spreadsheet tracking my BURNS score and made an effort to regularly write down when I have a bad thought and put it on trial to see if it fits any of these distortions.

That paired with exercise alone has made a big difference in my mood.

But that's just all anecdotal.


You can proxy your OpenAI calls through a quick Pipedream workflow.

Here's a proof of concept you can copy: https://pipedream.com/new?h=tch_OknfQd

The link makes a new unique API endpoint that proxies your OpenAI API credentials.

It just accepts a "prompt" argument in the HTTP request, but you can modify as needed.

If it does start to be abused, you can add frontend JWTs to check on this backend.


Pipedream has a similar paradigm, and Notion support as well.

https://pipedream.com/apps/notion

You can write your own "Sources" that can poll any API, it even includes deduping for free: https://pipedream.com/docs/components.

I build custom sources all of the time for scraping RSS feeds, WooCommerce stores, and then triggering low code workflows as if they are webhooks.


And Pipedream supports Svix[1] which means it has a top notch webhooks experience.

1: https://pipedream.com/apps/svix


Serious question, in an interconnected online marketplace how are you supposed to comply with these granular state and sometimes municipal laws.

Is there some kind of monitoring database or notification that you can subscribe to?

It seems very expensive for every single business to have a compliance lawyer specifically for this task of complying with each state's mandates to a tee.


> Serious question, in an interconnected online marketplace how are you supposed to comply with these granular state and sometimes municipal laws.

Identifying standards/laws and then combing through them for applicable requirements is just part of early product design.

When there are an overwhelming number of jurisdictions, start with the local ones in depth. Do a survey on the rest to get a flavor for what other areas are doing. Shape the product so that compliance is most likely already done or easily actionable when you get around to deep diving the other jurisdiction's rules.

I get that software-only products often don't have to do this, but it's not like it's a dark art. And while having a lawyer in the loop is important, you're rarely asking them to read the standard/regulation to you. They help you understand and digest when needed, and make sure you're doing it right.

Now, the topic at hand here is privacy which is a bit different. Finding a design that respects privacy and is probably fine nationwide is much less complex than morphing the product by jurisdiction to match the local minimum viable compliance.


I've argued in the past here on HN that your first employee should probably be an attorney--or at the very least have one on retainer, and got absolutely roasted for it. I still believe it though. How do you even know if your software product is legal everywhere you plan to distribute it? Are there any states that forbid what you are doing? Are all your dependency licenses really compatible? Are your logging practices legal in the EU? A single lawyer is not going to be a deep expert in Polish law, but he or she will be able to at least give general advice to keep the product from being dead on day one.

We laugh-complain about "ha ha the lawyers are designing our products now" but it kind of has to be the case in the complex legal environment businesses operate in.


Imagine 1000 competing companies in the same space.

500 have a team that looks like 3x developer, a design person, and a sales person.

500 have a team that is like 1x developer and a Lawyer

Which one wins? Which hits market first? Which is more successful?

An early company has to spend money on the core product. If you get wacked by a lawsuit and shut down in year 2, it's a cost of doing business and you go do something else.

There isn't enough time in the day or money for an early startup to get every single law perfect.

So to answer the OP: You just do your best, and correct when you screw up.


Developers these days get paid 2-3x as much as lawyers (until the later stages of a lawyer's career), so your 2-man startups appear to be starting with significantly less funding.

If you mean the choice is between a team of 3x developers and 1x lawyer vs 3x developers a designer and a salesperson, the former will win. Design matters very little at the earliest stages, and there's nothing for the salesperson to sell so they are most likely going to make promises to potential customers the developer's can't meet. And if they get any traction, CA or the EU will crack down on them for any privacy violations, which will kill the startup. Meanwhile, the former will chug along until they have a viable product that satisfies regulatory constraints, and acquihire the latter.

The days of regulatory arbitrage as a viable business model are over. There will be no more Ubers and AirBnBs. Governments have caught up to that scam and are significantly more willing to crack down on that now.


> Developers these days get paid 2-3x as much as lawyer

hhahahahahha

Not all of us live in the bay ;)

Also my bay layer was $450 an hour, have never met a developer that cost me that.

> The days of regulatory arbitrage as a viable business model are over. There will be no more Ubers and AirBnBs. Governments have caught up to that scam and are significantly more willing to crack down on that now.

that is pretty good too. Keep up the comments, this is a riot.


Sure and the teams that just blindly download packages from NPM will be the fastest to market.

They'll just also have massive supply-chain vulnerabilities :/

Unfortunately, being in business with a risk of going out of business is a better spot to be in than already being out of business.


This seems very obvious to me, so I find it weird people disagreed. But maybe that's correlated with the number of businesses I've started, which is 0.


Create a budget for a new startup. How much do you spend on each function?

It's not super hard to figure out why most startups don't have a lawyer.


> Create a budget for a new startup. How much do you spend on each function?

I have almost literally no idea; running a business is totally outside my interests.


K. The answer is you don't dump it all in legal if you can help it.


I work as a lawyer for a business that has lots of lawyers. Compliance with differing state laws is indeed very expensive. I would personally like to see a federal privacy statute that preempted these various and sometimes inconsistent state and local privacy laws.


> I would personally like to see a federal privacy statute that preempted these various and sometimes inconsistent state and local privacy laws.

It would have to be under the auspices of Interstate Commerce (to be legal under enumerated powers) but even then can Congress override State and local law?


Federal law is supreme, so yes, Congress can overrule any state or local law, and does so automatically if they have laws that disagree.


That's basically my reason for wanting all laws to be simplified to the point that an average person can remember at least half of them[0] in aggregate — if people don't know their rights or responsibilities without having to pay a lawyer, that feels unjust by my standards.

That said, there's no rule of nature that says running a business has to be cheap; it may not be optimal if all the laws in each marketplace are different, but I feel much less strongly about it — and not just because higher levels of government can unify and simplify when things get too much, nor just because I'm in Europe and one side of the border is all Rindfleischetikettierungsueberwachungsaufgabenuebertragungsgesetz and the other is Ustawa o przeniesieniu zadań z zakresu nadzoru nad etykietowaniem wołowiny because it's not just the law that isn't unified.

[0] of those laws that apply in general, at least; there's no need for normal people to know details of the accounting laws that apply to businesses unless the society really does want everyone to be a small business owner


> It seems very expensive for every single business to have a compliance lawyer specifically for this task of complying with each state's mandates to a tee.

It is. I've worked in healthcare, and it's not uncommon to have very specific requirements. However, there's no obligation to geolocate your customer beyond a simple request for their zip code, so it doesn't get too difficult from an engineering standpoint.


I think in practice, what ends up happening, is that small businesses don't really bother to comply while they fly under the radar. Or they just end up buying an e-commerce plugin that handles the minimum. We had one for GDPR and then they just added support for CCPA when that was a thing.


Right. The big guys with assets like Facebook and Google spend millions to comply.

The small guys duck under the radar.

It's kind of silly, but what happens when you have tons of laws. There is no realistic way a business of 1-10 people is going to be able to comply with every law in every state AND country they do business in.


I'm from the Northeast Ohio area, and also from a small town like East Palestine. The lack of response from Governor DeWine and malice to not declare a state of emergency is infuriating.

This rhymes with Flink Michigan's water crisis, and Obama "drinking" the water in a publicity stunt: https://www.factcheck.org/2019/03/posts-distort-facts-on-oba...

Vote democrat and only see the elite placate the working class. Vote republican and see unions and the facade of environmental protections decimated even further.

The fact this isn't covered in NYT or headlines on cable TV is insane. How is this unprecedented disaster affecting a major watershed not higher urgency than fucking balloons.


> The fact this isn't covered in NYT or headlines on cable TV is insane

Seriously, where is this idea coming from? Search the NYT, there's been coverage of the incident almost every day since it happened - not just the incident and its causes, but also on potential environmental fallout and meta-coverage of journalism and conspiracies around the whole thing. Many times it's been the top-line headline, at least when I am logged in.

I've seen continuous coverage on NPR, in fact the subject of this post is the current top headline on npr.org.

Every time I've been at a relative's who watches MSNBC I hear it brought up at least once during news hour.

My local paper a couple hundred miles away, despite thin coverage these days, has even had articles about what/any impact the fire has in our region.

The only event I've seen the has had anywhere close to the amount of coverage is the Turkey - Syria earthquake, which is a global level natural disaster that's killed 46,000 people and counting. I think that rightfully deserves coverage as well.

What national outlets are you following that you haven't seen coverage of this?


>Obama "drinking" the water in a publicity stunt

Yeah, they did an equally convincing photo op this time too. OhioEPA forgot to crop out the bottle of smartwater when they claimed to drink from the tap... and everyone immediately noticed.

https://twitter.com/flfitfactory/status/1627443706457427969


Grew up 30 miles west of East Palestine. My family is still there.

So sad to see another struggling Midwest town dealt a death blow.

It’s beautiful country even though it’s considered fly over and “blue collar” which is politically correct terminology for post-industrial poor.

The EPA has roots from the Cuyahoga River fires. There was great progress made cleaning up the river and even turning portions of Northeast Ohio into a national park.

Proud to be a Northeast Ohioan. There has been great steps forward since the industrial hay day.

This whole event makes me so sad. This community will most likely never recover and I’m hoping against hope there aren’t serious health consequences or birth defects from this. Though I highly doubt it.


Aha, that East Palestine.

https://en.m.wikipedia.org/wiki/East_Palestine,_Ohio

Took a while for me to figure out the connection with Midwest towns.


It's probably a good idea to have your family members visit a doctor to have blood drawn and tested for likely by-products of this accident and fire. To have a baseline proof of exposure could be handy later.


how the hell is Ohio midwest?


https://en.wikipedia.org/wiki/Midwestern_United_States - The Background section has the explanation.

I've always understood it as a sort of historical anchoring bias. Compared to the areas that were settled during colonial times, Ohio was indeed "West" in people's minds, just less so than the Pacific coast.


Wild. Learned something.


For a while there, the Mississippi river was the western extent of the USA.

In places like Ohio, you were midway there.


If you look at a map, it's clearly not. Not "mid", which would be from Chicago to Denver, and even more not "west", which would be west of Denver.

But "midwest" is a term for a region, and terms last longer than the reason for the term. When the term was coined, the US essentially ended at Chicago, and Ohio was midwest.

Hmm. I must be in a mellow mood today. I've made the exact same complaint you did when others said Pittsburgh was "midwest".


Standard cultural conventions of the locals I suppose. https://en.wikipedia.org/wiki/Midwestern_United_States


West of the applachians, east of the rockies. Midwest.


I always thought west of the Mississippi was Midwest.


(Rockies, Mississippi River) == Great Plains States?


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: