I agree that LLMs can be highly valuable when used appropriately. For me, their strengths lie primarily in semantic search (e.g., Perplexity.ai), rewriting and discussing overall code architecture. However, they often prove unreliable in specialized knowledge domains. If a user isn’t skilled in initializing and following up with effective prompts or discerning the usefulness from the responses, interactions can be hit-or-miss.
The robustness of LLMs—like the original ChatGPT or GPT-3.5—is still far from a level where domain novices can rely on them with confidence. This might change as models incorporate more first-order data (e.g., direct observations in physics, embodiment) and improve in causal reasoning and deductive logic.
I find it crucial to have critical voices like Gary Marcus to counterbalance the overwhelming hype perpetuated by social media enthusiasts and corporate PR—much of which the media tends to echo uncritically.
One of Marcus's demands is the need for a more neuro-symbolic approach to advance AI. Progress can’t come solely from "scaling up" models.
And it seems like he's right: All major ML companies seem to shift towards search-based algorithms (e.g., Q*) combined with reinforcement learning during inference time to explore the problem space, moving beyond mere "next-token prediction" training.
The robustness of LLMs—like the original ChatGPT or GPT-3.5—is still far from a level where domain novices can rely on them with confidence. This might change as models incorporate more first-order data (e.g., direct observations in physics, embodiment) and improve in causal reasoning and deductive logic.
I find it crucial to have critical voices like Gary Marcus to counterbalance the overwhelming hype perpetuated by social media enthusiasts and corporate PR—much of which the media tends to echo uncritically.
One of Marcus's demands is the need for a more neuro-symbolic approach to advance AI. Progress can’t come solely from "scaling up" models. And it seems like he's right: All major ML companies seem to shift towards search-based algorithms (e.g., Q*) combined with reinforcement learning during inference time to explore the problem space, moving beyond mere "next-token prediction" training.