Hacker News new | past | comments | ask | show | jobs | submit login
Launch HN: Meticulate (YC W24) – LLM pipelines for business research
100 points by JPalakapilly 9 months ago | hide | past | favorite | 38 comments
Hi HN, Meticulate gives finance professionals easy access to world class business research—think competitive landscapes, market sizings, customer segmentation etc. Here's a video of a competitive landscape generation in action (https://youtu.be/aJ-slHcp32c)and we’ve taken down the signup wall today so you can try it directly at https://meticulate.ai/.

Some background on “business research”: investment and consulting teams sink many hours a week into researching companies, markets, and products. This work is time-sensitive and exhausting, but crucial to big decisions like company acquisitions or pricing model changes.

At large financial services firms, much of this work is offshored to external providers, who charge thousands of dollars per project and are often slow and low-quality. Small teams lack the budget and consistent flow of work to employ these resources. We’re building an automation solution that brings a fast, easily accessible, and defendable research resource.

Meticulate uses LLMs to emulate analyst research processes. For example, to manually build a competitive landscape like this one: https://meticulate.ai/workflow/65dbfeec44da6238abaaa059, an analyst needs to spend ~2 hours digging through company websites, forums, and market reports. Meticulate replicates this same process of discovering, researching, and mapping companies using ~1500 LLM calls and ~500 webpages and database pulls, delivering results 50x faster at 50x less cost.

At each step, we use an LLM as an agent to run searches, select and summarize articles, devise frameworks of analysis, and make small decisions like ranking and sorting companies. Compared to approaches where an LLM is being used directly to answer questions, this lets us deliver results that (a) come from real time searches and (b) are traceable back to the original sources.

We’ve released two workflows: building competitive landscapes and market maps. We designed it with an investor running diligence on a company as the target use case but we’re seeing lots of other use cases that we didn’t originally have in mind—things like founders looking for alternative vendors for a product they’re purchasing; sales reps searching for more prospects like one they’ve already sold to; consultants trying to understand a new market they are unfamiliar with, and more.

The main challenges we’ve been overcoming are preventing quality degradation along multi-step LLM pipelines where an error on one step can propagate widely, and dealing with a wide range of data quality. We’re working hard on our next set of workflows and would love for you to give it a try at https://meticulate.ai and would appreciate feedback at any level!




I once had a business owner interview me and conclude that he wanted to hire me because he felt that I was a good thinker, someone who was, "real meticulate".

I told the boss that actually that is spelled meticulous. They said that he was pretty sure that it was spelled meticulate. I got the job offer, but I refuse the job. The thought of working as and inferior for somebody who would sucker down on the spelling "meticulate" was just way too much for me. What other harebrained bull** would they have me commit to?

Not sure why I bring that up here. It's a personal experience that makes me have no interest in this product whatsoever. Which is not rational. I guess I'm just a hater and I wanted to share this funny story while also maybe causing a loss of revenue to a random company I have nothing to do with.

Cheers!


It's ironic because you were the one harebraining over trivial bullshit.

We have millions of customers to serve and billions of dollars at stake, but you're going to nitpick over this guy's colloquial word misuse?

Come on.


If you have millions of customers and billions of dollars at stake, you better ensure that your CEO knows how to spell meticulous.

This person had so many red flags you could have called him Stalin. A bumpkin with missing teeth and a meth habit. I was applying as a mechanic in 2008. I didn't take the job because I didn't want to get f**ed around. So how about you give me a break? :wink:


Your attention to spelling further shows your meticulousness


I'm pretty sure it's actually hair brained


There's a Romanian joke about that:

De ce este fericit omul chel? Why is a bald man happy?

Pai, părul îi crește spre interior și îi gâdilă creierul! Well, his hair grows inwards and tickles his brain!


No, it's hare as in rabbit.


Woosh, probably :-)


> delivering results 50x faster at 50x less cost

What about quality of results? Are you measuring that too? Did you do so for the traditional reference practice? Using what sort of methodology? How did it your technique compare in quality? What kind of errors was it most likely to make? What techniques have you devised for spotting those errors? Are they the same kind of errors that users would experience when outsourcing? Are the errors easier or harder to spot for one than the other? Are they faster to remediate with one?

I see a clever concept but given the state of LLM's and the nature of how they work, I don't know that nominal cost and speed differences are really enough to sell on. Not for something "crucial to big business decisions." I'd want to know that my failure/miss rate is no worse than when outsourcing and that my net cost and time (including error identification and recovery) still end up ahead. I don't see either of those vital issues touched upon here.


This is a great point. We completely agree that high-quality results is essential for adoption. It's basically table stakes for any tool like this to be useful. We've had several versions of this tool that weren't quite "good enough" and never saw any real use. Our latest version seems to meet the first quality threshold for actual work use.

Our method of evaluating quality is not super systematic right now. For this competitive landscape task, we have a "test suite" of ~10 companies and for each we have a sort of "must-include", "should-include", "could-include" set of competitors that should be surfaced. We run these through our tool and others and look at precision and recall on the competitor sets.

In terms of errors, right now our results are a little noisy, since we're biased towards being exhaustive vs selective. There are obviously irrelevant companies in the results that no human would have ever included. Our users can fairly easily filter these out by reading the one sentence overviews of the companies but it's still not a great UX. Actively working on this.


I wonder if it's more about convincing yourself that it faithfully follows the same workflow an analyst would follow. It's always possible to miss stuff, so the best a person or a machine can do is be demonstrably methodical, it sounds like... and that is easier to test. Unless there is really some magic tacit step that human analyst perform to get better answers.


Well, human analysts get on the phone and ask people questions.

Not that an AI can’t do that too!

Though I may hang up…


I am sure most people never asked these questions to a human doing this research.


Most B2B focused AI tools will do 10x better if they pretend to be a normal human run company but just have the AI at the back.

Their clients want to know that the research report was written by a real person and not a bot.

But that doesn’t mean it actually has to be written by a real person and not a bot.


As somebody who have seen a lot of market research coming from "real humans"(TM) I'm pretty sure it's actually hard to make it worse. Maybe there are boutiques which make quality research, but average is abysmal.

I've been in a meeting with a senior executive, PhD, who argued that Total Addressable Market for a supply chain tracking software (basically just a database) is same as the market for the goods it tracks.

It is as if a startup which prints restaurant menus would claim the value of all food sold in restaurants as their TAM.

And this person was otherwise reasonable...

So, yeah, never underestimate the power of Natural Stupidity.

Perhaps


This was a huge problem when I was working on a similar product some years ago. The issue is that false positives or worse, false negatives, can be pretty catastrophic when you're thinking about valuing companies in particular. What's hard for LLMs (currently) is to decide what is most important or to infer the things that aren't explicitly stated.


Yeah when it comes to anything involving numbers there is absolutely no room for hallucination. It must be 100% accurate 100% of the time. No exceptions.


Have you ever seen the research coming out of some of the outsourcing shops that the OP discusses in the post? They are hard not living up to that standard. It's important to realize that this is input for the analyst at a fund or investment bank to do some more digging on the companies and in the process potentially discover more. This isn't going straight to the CEO to form the basis of an investment decision.


Congrats on the Launch HN!

I've heard a few investor types say something like "You know what's surprisingly fun? Popping an edible and making market maps"

Here is an example output: https://meticulate.ai/workflow/3b3fe891f16fc437acca87c0

It was really nice to go away for a few minutes and come back to this. Output is not perfect, but I wouldn't expect it to be at this stage.

I assume slide deck output is on the way?


Hahaha that reminds me of Erlich tripping in the desert trying to come up with one-liners for Pied Piper. Yeah definitely know our results are not perfect. We're going for being as exhaustive as possible right now so results can be noisy. And yes, slide generation is on the very near-term (next few days) roadmap :)


Congrats on the launch. This looks awesome.

I'm actually working with a number of companies who are exploring this space. Many of them are in the current YC batch. We're helping to provide the core business data, then we're exploring how we can leverage our scraping infrastructure in other ways to bring costs down.

I'm open to chat if you're interested: michael (a t) bigpicture.io


thanks! great to know, just emailed you!


This is so cool, congrats on the launch!

I have a friend who looked into doing something similar but they couldn't figure out a way to get the cost low enough. This was like a year ago so I'd guess it's much cheaper now and you could do something like fine-tuning a smaller domain specific model on GPT-4 outputs.

Any ballparks on pricing / cost? What models are ya'll using?


Thanks! That's fascinating, do you remember what kind of tasks they served?

Using GPT-4 and GPT-3.5 currently, and costs can be $1.50+ per request right now (have been benefiting from YC cloud credits!). Definitely steep at the moment but we expect costs to come down at least 10x over time.

Not super clear on pricing yet (only a few weeks post-launch)


I think it was similar... an agent finds a bunch of info on a private/public company to evaluate investment (automating some associate analyst work). TBH I'm not sure where they ended up but I know they had an interesting distribution channel lined up. Happy to connect if you want.

> $1.50

Thanks for sharing! I think they were ~10x that but hadn't done a ton of optimization yet. To me, having a swag at cost makes these tool demos a lot more interesting because you can start figuring out what types of businesses you can/can't build with them.


would love the connect to learn more about what they tried, I'm at wilbur at meticulate.ai

Definitely... we're lucky to be in an industry where there's a lot of money at stake


I tried it quickly on my company (some IT consulting multinational) to find competitors, and I was wondering if there was an easy way to do the following:

- for each company, find their main product lines and their respective market share (i.e. my firm does multiple things such as architecture, data governance, data engineering, data analysis, etc) but we're not necessarily the best on every type of service. I'm assuming that's a common pattern (e.g. some automotive companies are great at trucks but not the best at smaller cars, etc). - get basic financial stats (revenue, market share) - (bonus) get time-dependent stats (what was their revenue like 3 years ago ? What's the YoY growth ?)


This sounds a lot like what Grata does, although they've built a pretty impressive proprietary dataset


Congrats on the launch!

If I have want to help you in your roadmap- specifically around Find Companies, how can I contribute?


Thanks! shoot me an email at joseph(at)meticulate(dot)ai


Good job on the launch!


It couldn't recover from the task after the computer sleeps.


Do you plan to offer an API as an alternative to the UI linked?


not in the near term but happy to talk about what your use case might be if you want to shoot me an email at joseph(at)meticulate(dot)ai


Looks very promising. Will try


thanks! let us know what you think


I like it so far. The report's readability can be refined a bit. It's too noisy.


definitely, appreciate the feedback!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: