For Java applications, we built a structured logging library which would do a few things -
- Add OTel based instrumentation to generate traces
- Do salted hash of PII (injected in plain text by API Gateway in each request) like userid, etc to propagate internally to other downstream services via Baggage
- Inject all this context like trace-id and hashed PIIs into log
- Have Log4j and Logback Layout implementations to structure logs in JSON format
Logs are compressed and ingested to AWS S3 so it is also not expensive to store so much logs to S3.
AWS provides a tool called S3Select to search structured logs/info in S3. We built a Golang Cobra based cli tool, which is aware of the structure we have defined and allows us to search for logs in all possible ways, even with PII info even without saving.
In just 2 months, with 2 people we were able to build this stack and integrate to 100+ microservices and get rid of Cloudwatch. This not just saved us a lots of money on Cloudwatch side but also improved our capability to search to logs with a lot of context when issues happens.
hey, we're in pretty similar place logging wise, and I would really like to know more about your solution. If at all possible, I'd like to understand your rationale and implementation architecture more.
Sam would have been more apologetic or at least contrite in his tweet if it was hurting anyone. Same: Eric Schmidt was immediately positive, so presumably he knows. ES would never defend a guy who hid a leak.
Unless if, by “security” you mean OpenAI was used for military purposes, in which case: 100% Schmidt knew and supported and Sam might be proud of it.
But Ilya and Mira would have known about it too… Guess they did, told the board and things blew up fast.
When I read the heading then I was of the view that it is about how high the prices of Telehealth is but it was other way. Is it just me or anyone else also feels that the prices are pretty high? Does the article talks about Specialist or General Practitioners?
Disclaimer: I work as a Software Engineer for a Telehealth company
It can be kind of high in other contexts too in my experience. I've been using a place called adhdonline for my medication and the zoom meetings (every 3 months after the first month) are $169, no insurance accepted, and my insurance won't reimburse it either. It was a lifesaver being able to find this place since doing the whole procedure of getting medication in-person seemed too daunting, thanks to the adhd. I could find an in person place now that my insurance wouldn't discriminate against but this service is too convenient.
I noticed that people have already contributed great insights on readability and testability aspects which were my first thoughts as well on reading this blog.
However, I do believe that there is no one right answer to this argument. And the right answer is with that team who in the end have to read, write and maintain that code. The metrics that I collect with my co-workers who work on same code base as me are
* What is the cognitive load to grasp the code for members in the team?
* How easy is it to onboard a new member to this team?
* Are we able to move fast and have confidence in the code changes we make?
In my opinion, metrics like these are usually the ones most of us care about in the end.
so that team knows all future hire? I work on code bases that were written 15 years ago by a plethora of people in different imperative styles. where business requirements changed a lot over that 15 year period.
There's a lot of spaghetti code.
we found There's a strong correlation between method/function size and bugs.
There's not a lot of confidence because there's implicit mutable state and side effects all over the place.
Lately, Google has been killing products on spree. Today I read the news (1 month old) that now Firebase Dynamic Links are also being killed. Coincidently, in my current organisation we went live with some solution based on Firebase Dynamic Links recently with some partners. And now with this news, we have already started the discussion about it's migration.
A bigger discussion is that how easy is it for Google to pull the plug, then how is one expected to rely on Google for various cloud solutions they offer. Also recently Google had shut down it's domain service. Gergely had also written a blog about it https://blog.pragmaticengineer.com/google-domains-to-shut-do... .
In midst of all this chaos in Google Cloud offering, my questions are:
* Is your company concerned about these changes?
* If yes, then what is the plan of action you guys are planning?
* What are your view on this topic?
Yes. These actions just give me more fire as argument against every using something that is done by google. Be it GCP, Firebase or anything else, really.
Isn’t this going in circles? SaaS is completely normal in corporate America, but Google stands out for killing things. The question can’t reasonably be “saas or not” for most companies while “really google?” Is extremely par for the course.
Migrating SaaSes is a normal activity in corporate America and often wins the ROI calculation, but building on GCP is generally questionable at this point.
Sure, Google is "special" among large vendors covering a wide market (Amazon, Microsoft) that "fails fast".
But if you stick to long running products, even with Google, you are unlikely to have that issue. Eg. so many companies are relying on Gmail and Google Docs for their work: I don't think anyone feels they are going away anytime soon. Basic Google Cloud services (eg GKE) are likely to keep humming along fine.
But there are a number of smaller SaaSes that will show up with a specific niche product (like Firebase Dynamic Links here) that may be gone as quickly as their VC capital runs out.
Honestly it has come to the point that I will ask which cloud provider the company is using in an interview and I turned down a job offer once because of it.
For me it just isn't worth the risk of working somewhere that uses GCP. The high risk of there being financial issues when google announces something but also just the risk of that completely destroying a roadmap.
AWS provides a tool called S3Select to search structured logs/info in S3. We built a Golang Cobra based cli tool, which is aware of the structure we have defined and allows us to search for logs in all possible ways, even with PII info even without saving.
In just 2 months, with 2 people we were able to build this stack and integrate to 100+ microservices and get rid of Cloudwatch. This not just saved us a lots of money on Cloudwatch side but also improved our capability to search to logs with a lot of context when issues happens.