So I have to deal with both tools at my company. The reality is that it all feels so incomplete from a BI standpoint. Managers are throwing money at front end tools like Salesforce and Tableau, but the entire back end stack is still pretty much the same as 20-30 years ago (big expensive Oracle-ish databases).
I think the development of Python and Jupyter and other less known things like Vega are much more interesting. Python is today the only "glue code" that puts all of it together, from data to insights.
> the entire back end stack is still pretty much the same as 20-30 years ago (big expensive Oracle-ish databases).
Other than the expensive part, is it really such a bad thing? I feel like relational databases are a pretty good fit for a wide set of use cases and have a huge amount of tooling.
The businesses are coming with requests that require complex SQL on millions of records of data that normally is sitting in various sources (warehouse, salesforce, etc.). Unless you hire expensive data engineers, you can't do this type of work reliably. You can stick things together with expensive GUI oriented prep tools like Alteryx, but you pay in reliability and, quite frankly, sleep. And forget IT, IT is so stuck in their ways that you'd be waiting years for each analysis + you'd spend 10x what you should.
Isn't the problem space here genuinely complex in terms of business-complexity? Is there some better alternative that doesn't entail some other massive tradeoff such as managing your own servers, creating ingress mechanisms from multiple systems, building your own version of salesforce etc.?
In short, is there any solution that "does everything you could possibly want" while ensuring you _never_ need to hire a data engineer? This is a holy grail that I don't think exists.
You have to normalize data taken from various sources of various age and complexity. So you really have to understand the data. You also have to really understand the questions.
I've worked with (and on) lots of these tools and projects; the complexity is never in the frontend, it's dominated by getting the data, getting the data right and into the right format.
If all you want in the end is a good looking dashboard on a website then you might as well build it yourself; because of the cost structure that can even cost less than buying one of the BI frontend tools (there's not a lot of difference in development time, the the BI frontenders are more expensive because they are rarer and the licencing is high).
from my humble experience if you have a sales or product team keeps pumping out spreadsheets in weird formats you need someone dedicating a few hours to get a proper etl, and if they are constantly changing the format or adding new things you need a dedicate person just for that. Modern tools like Python or Power Query are not enough for this eternal war.
It's not that, it's the systems. 15 years ago I built a pretty sophisticated for its time data warehouse for a company that ran call centers. The amount of data that came off of the call systems was staggering, and the format arcane. Every vendor patch had the potential to wreck the ETL process. Then there was account data from clients, and other internal systems.
The people and their spreadsheets was the easy part to control.
Let’s say you have 20000 tables in total for a company. They are in 10 different databases. You have no overview over the data and no comments. You don’t have a starting point for where information x are.
Welcome to my reality.
Would I love a data architect and a domain expert in my team? Yeah.
Will I run around booking meetings with everyone that even hints at working with data like a headless hen? Yeah.
Is this the normal procedure for Data Scientists in big and old companies? More so than I would like.
Oh! And I forgot that the security department will constantly deny your access to data you need (until you force their hand).
Everything you mention is true and is compounded if the data healthcare related. Privacy concerns, data from different systems that claim to be the same. Preventing reidentification.
If you can get your data safely to S3, Athena can handle a lot of reporting and analysis use cases. The table or view definition can handle the normalization process. Full on ETL pipelines are sometimes (but not always) more engineering than necessary.
(Disclaimer: I work in data engineering at Amazon and use those tools in my day to day)
I am hiring one in Krakow! Seriously though, in a team of 10 business analysts I can barely afford 1 data engineer. Business analysts tend to cost less and also be more "business focuses", so they are an easier sell to management.
You seem to have an engineering problem, so hire engineers and perhaps fire some of those analysts. Don't make your business depend on someone else's tailored IT products, they will reap the profits you could be making.
Yes, I am a bit confused by that statement too. Isn't it good that complex tasks are handled by specialized people? Maybe it is my bias as an ex-big data engineer / current data scientist, but it seems to me that a lot of the tooling is is pretty simple as it can be (yes yes complacency is the enemy of good, I mean no obvious things to improve as low-hanging fruit)
Tableau has a data etl tool called “Prep” that helps with this problem. But it only goes so far. But I think that’s where the problem truly requires a data engineer.
There are plenty of modern day ETL tools like Funnel, Improvad or Dataddo to help with that part of the puzzle, though it does mean you have to pay another saas each month on top of Tableau.
Exactly. Instead of ETL, start writing your own Perl and various logical, reusable components. Roll your own ETL, however you want it, in a terminal. So what, you have to learn vim, big deal! Mouse driven interfaces are a huge part of the dysfunction.
Yea I've been a little confused here until I realized I would just write some bash, Python, Perl...etc script where some would advocate for complicated tools.
And after a few years you leave your job, a new person comes in and gets stuck with your script soup and lack of documentation.
Companies prefer well known products like Alteryx or Tableau because, despite the cost, it makes people easier to replace.
But i cant blame you for writing your own things. Im currently replacing a large SSIS-based etl proces with Python, because i'm sick of SSIS randomly breaking.
You make it sound as if this were a bad thing. RDBMS work well for many use cases. There are plenty of tools around to work with them. Good open source implementations exist.
The problem space (Business) is not complex, its incredibly simple.
Unfortunately the design patterns we use and the tooling is flawed.
It has been this way for at least 25 years and the RDBMS, languages, design patterns, and architectures ( 2-tier/ 3-tier) are the cause.
They make it simple to get started and even without knowing what you are doing you can easily churn out something that works if it is simple, doesn't change often, doesn't need to scale and deals with small amounts of data.
"It represents a quagmire which starts well, gets more complicated as time passes, and before long entraps its users in a commitment that has no clear demarcation point, no clear win conditions, and no clear exit strategy."
I feel like you are coming from an alternate universe. Nosql is the quick and easy thing to start with, then your needs become more and more complex and nosql just won't cut it anymore. Sure, nosql can scale and perform, but only if your needs are very specific and simple.
There are no major systems out there of even moderate complexity that aren't built on an rdbms.
Sorry I didn't explain it clearly. RDBMS are the reason we write business applications the way we do now. They are the root cause, swapping out RDBMS with NOSQL will solve nothing, because our languages, architectures, patterns, and libraries, how we even think about solving these problems all evolved on top of this and are flawed
That's the problem, business systems are not complex, they are incredibly simple, they are made complex by the way we think about and structure our models and then interact with them. We have the wrong boundaries, and the wrong languages
Would you be able to point me towards modelling approaches/boundaries/languages that would be more appropriate? I'd be interested to learn about better alternatives, as I don't yet see the big flaws in relational models
Counter-intuitively Datomic is in violent agreement with /u/rqmedes where he said "A better alternative is having the data, data model and business logic tightly bound in one place. Not separated in multiple "tiers"" – Datomic inverts/unbundles the standard database architecture such that the cached database index values are distributed out and co-located with your application code such that database queries are amortized to local cost. Immutability in the database is how this is possible without sacrificing strong consistency, basically if git were a database you end up at Datomic.
Unfortunately there is no real alternatives, its like operating systems, one or two system have so much momentum that using anything else becomes extremely difficult even when they are inferior in certain domains. see http://www.fixup.fi/misc/usenix-login-2015/login_oct15_02_ka...
I played with Denodo (data virtualization software) a couple years ago and thought it was pretty legit.
In theory, it could be used to provide that industrial strength abstraction layer between your Tableau/Looker/etc. and your bajillion weird and not-so-weird (RDBMS) data sources.
That would seem to make sense to me from the point of view of -- I would want my data visualization/analytics-type company to be able to concentrate on data visualization/analytics, not building some insane and never-ending data abstraction layer.
The part that surprised me was that Denodo could allegedly do a lot of smart data caching, thus speeding things up (esp hadoop-oriented data sources) and keeping costs down.
I'm guessing the other data virtualization providers can do similar.
I have had to work with the Denodo for the past 1+ year, a total nightmare. Data virtualization is a "good in theory" concept but "doesn't work in practice" reality. Going back to the original sources for each query doesn't work, it will always be slower than using a proper analytics data warehouse. Caching doesn't help because at that point you can just do ETL. Also Denodo itself is full of weird behaviors and bugs, my team collectively decided it's worth the most hate of all the "enterprise" tools we use. One thing Denodo is good for is as an "access layer", but then maybe PrestoDB would be worth a shot or maybe even just a sqlalchemy and python.
I don't understand why this gets downvoted. While it may lack context (with the aim of being controversial), it sparked a healthy amount discussion here!
My Salesforce clients have increasingly been considering Tableau. Tableau has great out-of-box discoverability of geo fields, dimensions and facts. The ability to auto-suggest a map of the U.S. based on data containing City/State is "magic" to power users.
The only barriers to Salesforce + Tableau adoption I noticed were cross-object JOINs and live vs cached data extracts.
Both issues were remedied by denormalizing the data prior to export. For example, a nightly flattened "view" of Opportunities with key related objects moved into columns.
Mulesoft is well suited to perfect the ETL challenges. Bringing them to the table could be a win for everyone.
Tableau might be friendly to analysts/end users, but its a piece of at the back end. The linux version they released is no more than a wrapper of original C code into JVM and thus causing huge performance issues and memory leaks. Being that said GL CRM
> I think the development of Python and Jupyter and other less known things like Vega are much more interesting.
In that case you may be interested in Dash (dash.plot.ly). It’s a free and open source library that you can use to create dashboards online with Python only.
We love Dash on our team, anything more than a Tableau dashboard goes into Dash. You can basically just treat it as a Flask app.
We write our back ends with FastAPI[1], which is usually just a wrapper around our ML models. Then serve both Dash and FastAPI with gunicorn. The backend is provided the uvicorn[2] worker class with the gunicorn -k arg[3] to greatly increase the speed as well.
For personal projects you can use this stack in GCP's AppEngine standard environment to basically host your (relatively low traffic) apps for free.
That's not totally accurate... If you use our SaaS Chart Studio product then yes, but otherwise (i.e. in 'offline' mode) you can use our Python, R and Javascript libraries as much as you like: they're MIT licensed.
I'd like to add that Dash helped me grok React far quicker than I'd expected. I code in a lot of React now and I'm only here because of Dash. Thank you for your hard work!
The backend has also evolved massively from Hbase to the cloud-powered data warehouses. We have the ability to ingest and query petabytes with single-second delays now. There's also on-demand querying like Presto/Drill/Dremio, ETL systems like CBT, and the growing space of "data lineage" for seeing how data is connected and has evolved over time.
The real issue has always been the organizational problems of larger teams and companies as data gets split into multiple silos and needs ETL and cleanup before it's useful. The new abilities we have gained have increased the complexity and scale which can lead to new challenges, but the tools are definitely getting better every day.
If you have to write code for this stuff 90% of people won't be able to analyze things, and 90% of analyses won't happen because they won't be worth the time.
I found the same thing with MicroStrategy. I spent a lot of time reverse engineering what I could from MicroStrategy jars to expose additional functions in their plugin interface (which is so incomplete it shouldn't be advertised). But the reality is its 20+ year old system with front end updates, can only put so many band aids on it.
I think the only thing keeping MicroStategy alive is its cube functionality and the businesses who have invested to much into it.
Its a javascript graphing library.
We've started using it at work a bit for our online dynamic graphs. Its a graphic grammar on top of the d3 js library.
We seem to prefer the lite version, which is simplier.
If you look at the examples, you can click a button a go to a dynamic editor which we rather like.[1]
Vega is nice for its declarative, portable nature. Also, it makes certain things very easy such as embedding interactive Vega charts and generating images of charts without needing something like PhantomJS.
Hey - I'm hacking on a product which allows you to build interactive browser-based apps/dashboards from Python and dataset blocks. You can return altair (vega) plots from your Python and they'll get rendered in the app, and we're just adding the ability to import existing Jupyter notebooks this week.
It's still really early, but feel free to have a play and create an app. Here is an example app examining using the Prophet forecasting library: https://nstack.com/apps/rdA647Q/
I'd love any feedback, and if you'd like to chat to learn more, reach out to me on leo@nstack.com.
I can relate. The sales pitch is still “so easy anyone can do it”. As a result, corporate money is thrown after tools that enable business folks to build dashboards...which isn’t a good use of their time. :). In truth, they need good data folks working with them. I guess that doesn’t make for a sexy product sales pitch.
I think the counterargument is that it's not a good use of their time to convince someone to build it for them, when they can finally do it themselves in half the time :-)
Lindy effect. If it has survived that long it probably will survive in future too. Unless the quality of individuals working in non-tech goes significantly up in tech savvyness I find it hard that it will change.
When I look at Jupyter notebooks, I ask myself why is anyone bother with ootb reporting tools or growing their own? Jupyter notebooks are the right mix of customizable and self-documenting.
The kinds of analysis and visualizations you can build in 5 minutes with Tableau (and the ability to explore the space of possible analyses and visualizations) would take hours of futzing to reproduce with python.
You can build a basic report in 5 minutes. Then you’ll spend hours tweaking it and making all the changes the boss wants. And then hours more discovering there’s things you simply can’t do (but you’re boss won’t accept that).
And next week you will have to do it all again, because it’s all manual.
How many business users are going to use Juypter notebooks? Meanwhile, someone with basic computing experience can create in-depth reports with Tableau in less than an hour.
> How many business users are going to use Juypter notebooks?
None, because it's too much programming for IT to let business people have access to it, and it's not disguised as an office productivity app the way Excel is.
If they had access to it and had basic training on it that anyone already competent in any vaguely quantitative domain could handle, plenty of them could and would.
At least judging by my experience with SQL shells and similar told that are both less powerful and less friendly than Jupyter + Python, and yet plenty of business people used them productively in enterprise environments (often right up until IT ripped it from their hands.)
Python is making some inroads in shops that have been using SAS and/or R, but it’s a hard sell. Jupyter is even harder to push because of the server element.
SAS and R are pretty common business user tools that are basically the same thing as Juypter. Tableau is nice, but you generally need to use something else to prep data for it whether that's an ETL process IT sets up or something manual that an analyst publishes.
The backend stack of both make development like swimming in glue. The same managers you are referring to cut us over to these tools and we subsequently experienced a massive decline in our business.
Are you referring to APM/infrastructure monitoring, etc.? I was curious to find out if you're thinking BI capabilities need to be just as powerful at the backend as these frontend tools.
In business analytics backend > frontend. If you are a business analyst you want all your data at your finger tips + the ability of running somewhat complex transformations and calculations. You also need to be able to validate data with your own rules, because you can't always trust data from warehouses and other sources. Then you want to take all your work and schedule it, so it runs automatically and refreshes all the reports depending on it. Finally, you want triggers on data changes/updates and good looking email notifications with screenshots, attachments, etc.. Of all of the above, end users might only see the dashboards, pdf's or Excel's on the frontend part. If you have a solid backend, you will be able to serve your customers at multiple times the speed and with a lot more insightful reports. At that point the difference between a Tableau dashboard (which does look good) and, say, Superset (which does not look as good) is somewhat negligible.
Yeah, it's certainly more than just a CRUD, and I agree it's awful. I never had to use it in the early years, so I don't know if it's that their original platform was poorly made, or that the accumulation of bloat & bolt-ons is what has made it this way.
Yeah but to someone who is in to working directly with RDBMSs, Salesforce is 100% front end. It's like how CEs think C is high level, but web developers think C is low level.
> The goal of the Vega project is to promote an ecosystem of usable and interoperable tools, supporting use cases ranging from exploratory data analysis to effective communication via custom visualization design.[0]
Wow, Looker last week and now Tableau. I’m only 90 days in to running the open source equivalent inside GitLab, and these big revenue multiples are telling a fascinating story. I would’ve loved to be a fly on the wall for this negotiation, what a great outcome for Tableau shareholders!
With all that’s happening we’re definitely looking to pick up the pace, and would love to work with more contributors on the free open source alternative at Meltano (www.meltano.com)
Also second this, product guy here who has been using Tableau for years. Curious about Meltano. Went through some of the site / installation guide / videos and then the gitlab issues log and then found myself back here :)
Would like to see a few simple, visual stories about how one could derive business value from meltano, ideally real-life use-cases but if you dont have those yet just make them up.
Thank you for taking a look, you're right and we are working on it right now. This is perhaps a little more attention than we expected for our tiny 5 person project, and lights a great fire under us!
Here we all are in the comments complaining about how all the focus is on front end instead of fixing the back end, and one of the first comments to a legit alternative is "yeah so what's it look like?" !!
There are some great tools in this space, definitely worth exploring.
Our vision is to glue the steps from ETL to dashboard together in an end-to-end solution. We pick whatever we consider best in class and integrate it. So far, we've got Singer, DBT, Jupyter Notebooks and Apache Airflow and we're using VueJS for both the product UI and our website.
thanks for the product. I checked Singer, DBT, both of which are great idea and product. I build one very similar (conceptually) to DBT in my last data project. (I would not if I knew dbt is there).
Airflow is too heavy weight for me, I use Scons to do the workflow management.
meltano must do a lot of work to integrate all this together. I wonder what is the general user experience is. To me, seems too heavy weight.
Back when I was at a company that had frequent ETL needs, we tried a few, and eventually settled on Rhino ETL. It's not a pretty drag-n-drop tool like some of the others but it was simple and (therefore) worked really well.
I did some integration work with the venerable Pentaho project some years ago. It's got both an Enterprise edition and a Community edition that's open source with tools including ETL and more. https://wiki.pentaho.com/ looks like the best place to start these days.
Funny coincidence: I was reading about you in relation to Mattermark, then found your blog post on Meltano, and now I find your comment on HN. You're everywhere! :)
Would it be possible to chat with you about your past experience at Mattermark? I am trying to answer a few questions (not related to the company, but related to VCs and investing). If so, my HN username @ gmail. Thanks!
Have not decided yet, but I’m leaning toward consulting services and possibly even marketplace. There are so many more independent analysts, data engineers, and other experts/specialists who could provide a service network beyond anything we could hire internally.
It’s still very TBD while we get the product to a place where those opportunities start be more emergent. We definitely expect it to be an open core model though, similar to GitLab.
By remaining self-hosted we avoid the big expense (and risk) of storing users data, and they can pick whatever cloud they want. Our team is 5 core members working at GitLab, and we have about a dozen contributors. So it’s kind of a startup within a late stage “startup” (unicorn).
The work you are doing at Meltano is simply brilliant. It needs to be done and if done well I have no doubt you will be very successful. I plan to use it.
One thing I've had to remind myself of - coming from a CS/Engineering background similar to most folks on HN (I'm guessing) - is that there are 2 types of people: Those who program and those who don't.
To me, Salesforce looks like a big shared Excel file with a bunch of sheets. Tableau... well I can do the same thing with some scripts or spin up a web server.
To others, this tech is just magical. Pay the money, do the integration and it just works... And clearly people will pay a lot of money for things that "just work".
>well I can do the same thing with some scripts or spin up a web server.
That's a massive waste of your time and effort. The maintenance costs become massive as well should you choose to create your own system. At the VERY least you should leverage existing free open-source tools such as Metabase or Superset or Dash, or free tools such as Google Data Studio or Mode Analytics, if you're not going to spend cash to get a tool like Periscope Data/Looker/Tableau. I mean this gently, but you likely underestimate the complexity of a reliable reporting/analytics infrastructure. Think about it this way - these tools are either collaborated on by a large open-source talent pool, or are created by teams of dedicated software engineers just as talented as you.
I've worked with quite a few companies in an analytics consulting type role, and your "I can do the same thing with scripts" statement is one I've heard countless times. The long-term maintenance costs and technical debt (and "rigidity cost") of rolling-your-own analytics far outweighs the cost of a true analytics platform.
If you decide to roll-your-own anyway, look at tools like DBT and Airflow to reduce long-term maintenance costs.
Yeah I work at a company in the analytics space and see that all the time. It peaks the curiosity of people who are software developers (yet their core competency at their job is something else). They think its a fun work side-project and go after it. Write some python scripts to do ETL and process the data...make a backend with pg, a web server, then do charts in d3.js.
A year later they have a bunch of nice demos to their bosses but nothing that they can actually use in production because it crashes, there's no UI for interactive queries, no reports for people in the business groups, no user management etc. Then they drop it because they're busy with their actual job. So the cost of that engineers time to do something that didn't work was about $20-30k over a year. While the product they could actually use in production was around the same price.
Out of genuine interest and perhaps even a need for such a solution, is there anything that does what Metabase/Superset/Dash/GDS/ModeAnalytics does but for realtime datastreams? For instance, parsing and recording and visualizing the events coming from a websockets or some event queue/bus?
Yes perspective is built for streaming data - https://perspective.finos.org. Open source streaming pivoting engine that operates in the browser (using wasm in a webworker, with integration with Apache arrow for ingest of binary data strrams off websockets)
I believe you'd want a tool such as Grafana (which I believe is free/open source), which I have seen eng teams implement for realtime streaming. There is also Kibana which I am less familiar with.
Those all work for that already, because they're just frontends to whatever backend system you're using. You a database like BigQuery with real-time streaming inserts and rerun your queries for results.
If you want analytics directly on the stream, then there are plugins available to support reading the query results of something like Kinesis Analytics or Confluent's KSQL.
You might want to check Striim - it wraps kafka + streaming transformers with a scripting language, and a visual pipeline design tool. It then offers some really nice dashboards with a real-time “feel”.
Not affiliated in any way but I took it for a spin and was happy to see how much could get done with it in just a few hours and without prior knowledge.
How does Tableau make your reporting more reliable? My understanding is that Tableau can hook into different data sources -- like your warehouse and SalesForce, for example. Then you can write some kind of SQL to generate charts.
The auto-chart generation is nice. But what about Tableau makes it more likely to be accurate? Aren't you just as likely to make an error on the SQL in Tableau than if you didn't use Tableau?
I never said that about tableau in particular, which is why I listed half a dozen analytics platform solutions. Using any of those makes for a more powerful, flexible, sharable, usable tool than one engineer's self-rolled internal webpage with "analytics". Nothing special about Tableau in particular, in fact, I categorically prefer SQL-based visualization tools over Tableau.
The only exception to "using any of these is better than creating your own" is large companies like Google and Facebook, where they have entire teams of engineers who are dedicated to creating an in-house SQL+Visualization tools. It is absolute hubris for one engineer to think they can make a robust analytics platform!
Tableau is a GUI first. It's not designed for SQL based access and barely supports it other than as a type of custom datasource.
Since people aren't typing code, it can be more accurate to use, and it provides visual results beyond just a table that can be useful in detecting anomalies in your data.
As an engineer, this has really fascinated me. Tableau, Salesforce, Excel, etc. were things that never made a significant amount of sense to me. I thought mostly of Salesforce as a CRUD app (which it is!), Tableau as d3 with nicer ergonomics, and Excel as... well... something I never understood.
If this is something you guys are interested in, I started a company called Retool (https://tryretool.com) that is essentially Excel for developers. Imagine if every Excel cell — instead of being a cell — were instead a React component. So you drag and drop these components around, and you can connect them to any back-end datasource (postgres, APIs, etc.). So you could drag on a table and have it pull data from `select * from users` from postgres, and then drag on a button and have it `POST` the selected row back to your API, in order to ban a particular user. The goal is to let end users build CRUD apps (like Salesforce) around their existing datasources quickly.
If you guys have any feedback... I'd really appreciate it. We're just starting out, and really curious to get any feedback from developers. Thanks!
When I looked into this a few months ago, your pricing didn’t make sense to me. You’re saving developer time, but charging on user time. So I’ve got 800 people in this org and I could either pay a shit ton to use retool or just build it from scratch. The difference between both of these is my time so it makes sense I’d pay you for that.
That made it not viable for me. Building from scratch was way cheaper with Upwork. Anyway, Product was cool.
Thanks for the feedback — that's really helpful. I agree that our pricing isn't good for a large org that is looking to build a tool or two (instead of say, a few hundred).
Do you mind if I email you? (I can't see your email in your profile, but my email is david@retool.in if you're interested in reaching out.) I'm really curious — just to learn — what kind of pricing plan might work for you. Would a per-app model work for you, for example? Thanks!
I would have to delve deeper to give actually useful feedback, but I wanted to say from a presentational point of view, it looks great, well done for putting something like this together
Unrelated to the tool, your homepage is absolutely gorgeous. I imagine the rest of your execution is along those lines, which is to say - excellent. Well done!
To put a finer point on it, they'll pay a lot of money for tools that require less-skilled labor to operate. They will gladly spend a lot of money on Tableau to avoid paying the salary premium to have "those who program" build them an analytics system. It costs much less to hire business analysts who don't program (aside from SQL).
>is that there are 2 types of people: Those who program and those who don't.
Is that fully accurate, though?
Where does that leave data scientists / data analysts? I know SQL very well, and I know python's data stack (numpy, pandas, matplotlib, plotly, seaborn, various stats toolkits). I have a strong understanding of the "programming ecosystem" i.e. concepts, terms, definitions, and so on. I understand (basic) computer architecture, I've used and am familiar with (basic) shell/terminal, and services like Docker/Heroku on the command line, and can certainly use GUI cloud tools for AWS, GCP, etc. I can read and understand code and how systems fit together. I've worked alongside engineers of all types.
But I'm not a software engineer. I don't tell people I "program" because my strongest skill is SQL and generally people do not refer to that as "programming".
you are a programmer. At least in my book. You may not be experienced in architecting software projects, or writing highly maintainable code, or dealing with niche frameworks and tools (gui frameworks, network systems). But that is just a lack of training, interest, opportunity, etc...
This is nice to hear. Truth be told, I'd like to get some professional software engineering experience. The problem for me is:
1. It's hard to get considered to be a SWE in the first place when your job titles are more in the realm of data analyst. They'll toss out my resume for a fresh grad, much less someone with experience, without a second thought.
2. If I were to make the switch I'd likely have to start at level 0 on the scale ladder. I've already career changed once into tech, and at this point I do not wish to "reset" my experience another time
"Data Analyst" is often programmer lite. If you've been doing that work for a few years you could easily transition into a full time SWE role. If you really need a confidence boost and/or some on-trend training, consider doing a full stack bootcamp.
Yea no this is wayyyy underestimating the costs of data engineering. I can tell you from my very limited experience some queries I've seen, on some systems and some datasets, have estimated TTLs in YEARS -- and that's the difference between companies that invest in tooling and infrastructure (and big O analysis during system design) and those that don't. And you will invest a good deal of effort and time and money before you realize it, it's like the perfect ambush.
Never start a land war in Russia, and never neglect your data infrastructure if data is in any way a key business differentiator/fundamental in your market.
And this attitude right here perfectly exemplifies “NIH syndrome”. Beware if you work somewhere and a colleague suggests that some homegrown hacked-together non-feature-complete solution would be competitive and “cost less” than the professional product.
Wait until a manager asks you to do something that Salesforce was not built for and IT tells you a back office resource can do it. None of those of the shelves enterprise solutions just work, they all need some degree of customization and integration. I am struggling sometimes to justify the cost of a vendor solution + customization versus just rolling out my own or an open source one that can be more easily built on top of.
Massive customization, and then you are bound by this broke ass Object model in which to get it all done, between Apex and VisualForce nauseating crap.
I don't want to pay hundreds of thousands of dollars for the right to do write database driven web pages.
I thought that the point of Tableau was to provide a tool that end-users (who can't program) could use to interactively explore their data. That's not something you can replicate with a bunch of scripts.
I can see how Tableau would be great if you are in a situation where you can have business guys creating their dashboards and exploring their data.
The issue in practice is almost always getting the data in a workable state so that you can manipulate it easily in Tableau. In my experience in smaller and mid-sized places, Tableau tasks get punted to analytics and data science, because they are needed anyway to get and transform the data in the first place. And these people usually prefer and are capable of using more technical tools than Tableau. I know I would rather use Shiny or Dash.
Maybe that's not a difficult problem in larger corps.
If you just want a chart based on a DB query, then yes, it's a waste of time and money. The value of a lot of these products is that they have query/charting builders, which takes a lot more effort to implement. These days a lot of business aren't willing to accept analytics being controlled by a single person or team, as they're used to anyone being able to do their own custom thing.
Probably not. From his comment I doubt he's even used Tableau (and I say this as a programmer who worked on something similar - https://perspective.finos.org - and yet I still find myself turning to tableau pretty often).
We were getting ready to do a tableau api/integration with our software and the licensing came out to about $100k for two server dev licences and six client licenses.
Ah of course. Just a few few weeks of dev time to build the same thing a multi-billion dollar company sells. Then after a few weeks of dev time you find out you're missing about 80% of the features that makes tableau useful. After that few weeks your manager asks you how the project is going and you just mutter something about how you should probably just buy looker or tableau.
Nah, these sorts of folks are far too obdurate and prideful to ever willingly admit defeat. More likely they would keep telling the manager that they just need a few more weeks time to finish it while they frantically keep rolling that Sisyphean boulder up the hill.
Except it's not just a few weeks of dev time that makes up the overall cost. Consider infrastructure, maintenance, updates, support, training, etc,. Those things start to add up and you don't get the benefit of scale/community if you do it on your own. There's also the opportunity cost of building your own system when you could buy something existing and use that time to work on other things.
So how many seats are needed to hire somebody who creates a custom shiny + R app that does exactly what you want? How big is the average company that makes use of tableau etc.? Let's do the math ...
It’s 2019. Are we seriously still trying to argue that homegrown solutions are overall more cost effective than SaaS? Are we still of the opinion that developing tooling outside of your company’s core competencies is somehow adding value?
Absolutely. There are tons of people I encounter every single day that seem to think it is cheaper to build everything inhouse. I think it comes from a complete misunderstanding of how business delivers value coupled with RMS-like distrust of anything they cannot completely crack the lid open and mess around with. Mostly the former, some of the later.
Well it's probably a question of perspective and problem domain. Also my main concern is precision and correctness. Quite often you can choose between (1) customized tools that achieve almost what you want them to do, but often the people who implemented or customized them didn't fully understand the problem and did things wrong without anyone noticing for a long time, or (2) custom tools that do exactly what you want, that were tested by somebody who cares.
I actually felt the same way about Salesforce until about a year ago when my company bought it. It certainly has a duplo blocks side that let's non technical users do complex things (aka Excel) but their backed is extensible through a language that is more or less Java for custom handlers/queries/etc and they are switching their front end to W3C compliant web components for developer maintainability after getting burned by rolling their own front end framework in the early angular 1 era. They have super tight integration with heroku postgres dbs (because they own them) and have some other best technical tricks up their sleeve to the point that I my opinion has changed quite a bit.
This seems really high for a company without earnings and a weird growth curve. Their ticker is cool and maybe sales force wants to be DATA on nasdaq.
Otherwise, it will be hard to justify this high markup for a tool company.
It will be awesome if Salesforce can adjust their model and make Tableau spit out D3. Their desktop tool is nice for designing, but their server components seem frequently unnecessary for running the visualization. The catch is that creating serverless dynamic visualizations isn’t all that money-making and the cool UI/UX design tool is outside of OSS’ wheelhouse.
A lot of companies use Tableau to visualize their data. In the future if they want to use Tableau they (speculation) might have to host their data in SalesForce and/or be directed to use a number of other SalesForce features if they want the latest coolest dashboards. To me this acquisition seems be a play toward growing customer usage of SalesForce licensed functions and data migration to their cloud, rather than based on the current revenue stream. The lock-in is strong.
SalesForce has been pushing Einstein Analytics recently. I haven't used it, but I do see that moving an organization from Tableau to Einstein has a lot of costs involved so this would be a hard sell in many places. Having them both under one roof means they're able to bring a bunch of people across to their cloud and now that license revenue year over year is theirs with the additional data lock-in.
As someone that really dislikes vendor-forced lock-in and generally dislikes the way SalesForce controls your data, rate limits, maxes, licensing, etc, this move is about even more control up your stack that will seem like a "no brainer" to decisionmakers, which is dangerous. That said, I'm sure it will work well for some organizations.
EDIT: I would also love to see them spit out D3 or other open visual but then they'd be losing control of the secret sauce and the requirement for a license. Not sure there is an incentive to go that route.
My company is using Einstein Analytics, however, other than the Salesforce integration has been a terrible disappointing experience. Loading data is a pain and has to be done to a custom API. Of course Einstein analytics need to have it is own language called SAQL resulting in a steep learning curve.
yeah i put in another comment that this acquisition is a vote of non confidence in Einstein Analytics. You may see it phased out in the next few years.
Yeah SalesForce NEEDS Tableau. It is like a missing puzzle piece in their suite. Einstein is essentially enhanced reporting for SalesForce even though they sell it as an analytics tool.
That sounds right. This acquisition maybe gets rid of the curve there and for orgs like mine that use Tableau and SalesForce it sounds simpler / easier in future (Einstein has conversion and learning costs as you say). However, that comes with a cost - loss of any freedom to use your data without worrying about arbitrary license changes and cost. The way they structure the licensing will be interesting since they’ll have leverage over any customers.
Thanks, this is a good tool. There’s a few libs that spit out d3, but the UI isn’t as usable. Tableau is popular with the Excel crowd that I’ve been able to work with.
Precisely what I thought. It looks like their annual revenue is ~ $1 billion, placing this price around 15x annual revenue.
However, Looker has about $131 million in revenue, so their purchase price was an even higher 20x annual revenue.
My conclusion is that these acquisitions are much less about sales revenue and much more about filling strategic holes in product offerings, and I can only assume it's a sellers market in that area.
Multiplers don't scale linearly. The lower your valuation, the more possible it is to get a higher multiple for various reasons (cash, # of bidders in the market, etc).
At $1B/year in revenue there aren't a lot of companies that can realistically acquire you. At $131MM/year, there are.
I have worked with some companies that love Tableau so much, they use the desktop and server versions a lot. The licensing costs are high, but it does the job.
Because they are the clear market leader in the category, have been for a long time, only recently falling behind Microsoft. (within the last 2 years of their 20 years as a company.) Give it a year or two to integrate into Salesforce and they will be neck and neck. This is just as much a bailout to Tableau to save them from being dwarfed by Microsoft.
Or maybe neither, and just a weird factoid about a corporate acquisition that wasn't indicative of any larger market trends. (All possibilities now hedged).
Last year they acuired Empirical Systems, "an eight-person artificial intelligence startup based in Cambridge, Mass" [1].
However, 'Tableau Research' [2] has existed for years and its researchers regularly publish at major academic visualisation conferences like IEEE VIS (InfoVis/VAST) and EuroVis (see [3]).
Tableau worksheets and dashboards are proprietary and need either Tableau Reader on the client’s desktop or Tableau Server to run proprietary server side stuff for the js libraries to draw it. This is expensive to license their server and limited in how you can host. In many situations the data is small enough to send to the client.
It would be nice if tableau would just generate static content that could be hosted anywhere.
There’s not a client tool for d3 as nice as tableau. I work with lots of scientists who learned tableau but aren’t really programmers and can’t figure out d3 or other libraries.
Sort of. Salesforce is more interested in selling cloud licenses and space. They push proprietary apps and such, but maybe they won’t care so much about hosting tableau server specifically.
I held Salesforce stock from 2016-18 and it helped us just buy our first home. They have incredible lock-in with a very large and diverse customer base. They now have to defend themselves against Google's purchase of Looker, and the rise of PowerBI.
Are they serving the same market as Google? It seems as though the two are serving different use cases, both of which need data viz. Salesforce gives you a sales database and a front end out of the box, and you can tweak it and add to it with add ons. Google gives you compute and storage, but the application logic is all owned by you.
That's what makes this seem like a strange deal to me. There's pretty much 0 overlap with Google on CRM, and the next comparable tech stack that does compete with them is MS Dynamics CRM combined with Power BI. But Dynamics doesn't appear to be a growing threat for CRM marketshare.
That said, Salesforce (based on my usage ~4 years ago) has truly awful baked-in BI and analytics, necessitating third party products and data engineering to fill that gap. Tableau will fix that, but I'm staggered at the price.
Salesforce is a sales organization par excellence. They sell whatever they have, be it social media monitoring, CPQ, marketing automation, Heroku, Jigsaw data, IoT etc. they develop deep wallet share in large accounts and are constantly cross-selling. Their core audience is business people, and that is in stark contrast with GCP which is almost entirely dev focused. I’m not saying CRM is an afterthought but they have plenty of accounts that don’t use their CRM.
That said, Google has deep inroads with their apps suite, and it’s really a race to have their tech run business processes. That and their ecosystem like Insightly are making big inroads to CRM’s SBM market.
Their analytics is shit, but I think the important thing here is that they know their audience very well, and business people love Tableau.
They bought $15B worth of sales leads for cross selling their portfolio of services.
So is Google making a play for Saleforce’s core business? I don’t know a lot about the details of what Salesforce does but from my view it looks like there is not that much overlap
As I think about it, I don't see this as a play for business or much of direct competition at all, they simply both had strategic gaps, but for very different reasons, in their BI/Analytics offerings.
Interestingly, I think an earlier version of the title had "in all-stock deal", but I think the mods removed it. I wish they hadn't, as that's an important detail for the reasons you mention. (Also, ask James Baker, who was convinced by Goldman Sachs to sell Dragon Naturally Speaking in an all-stock deal ... of a company that turned out to be the Belgian Enron.)
I was in speech recognition tech during that time. It all tanked, but Dragon got screwed the hardest. ScanSoft (now Nuance) made out like a bandit with all the tech fire sales post-bust.
Yeah, I can't imagine how Baker (and all the employee-shareholders) must have felt about being advised not to do the standard half-cash-half-stock purchase and then turn out to be a fraud so soon.
My guess is that Salesforce was also in the race to acquire Looker, and since that fell through they picked up Tableau because they had sunk the work into acquiring a data visualization company. The timing is uncanny, but the price tag is vastly different.
First, how do you know what all institutional investors are thinking?
Second, Sales and FCF straddle Earnings on the income statement. A focus on EV/Sales suggests that investors are optimistic about growth and ignoring the spending required to get there. A focus on EV/FCF suggests that investors are optimistic about increased efficiency and cutting costs.
I love this. Mulesoft, Tableau -- there's a very clear strategy here for Salesforce. You're running all of your customer-facing operations on Salesforce, so why not also integrate the BI stack into everything?
I've worked with a lot of companies who spend months (if not years) integrating their data into a few disparate systems... The finance team has one system (and underlying data lake), the commerce team another, the marketing team another... If Salesforce thinks they can run the entire underlying data infrastructure in addition to the actual customer-facing functions, then this is a smart play.
If Salesforce wants to run my entire underlying data infrastructure, I think we’re in trouble. I see this more of a land grab, with SaaS vendors trying to push their own BI stacks and create an even tighter era of lock-in. I don’t think it is necessarily a good thing.
Data is an asset and liability - when somebody else has all of yours in their proprietary platform and under the control of their cloud, that is a scary proposition to me.
Fair enough if you are running your infrastructure on open standard tech and common cloud platforms, but not locked away in Salesforce.
This is reminding me of the behavior of other large orgs, like Oracle.
On the flip side, if Salesforce does not own its own BI stack, their customer might go to a different vendor that has better integration (i.e.: Microsoft PowerBI).
It’s a risky play. At my enterprise company, we’re not allowed to use a single vendor for everything, we explicitly must use different companies if a favoured vendor hits a certain spend threshold.
This is to prevent cost overruns and solution capture, where every solution to your company’s problems becomes “give it to X vendor” and then X vendor kills a product line and you’re toast.
Salesforce needs to be careful or else they’ll hit that threshold where companies don’t want to use them because you as a client are too small. Google is facing this problem right now.
Is this somewhat common? It makes intuitive sense, but the big 3 or 4 or whatever number it is (IBM, P&Y, etc) consultancies have "use us for everything" as their explicit strategy / part of their sales pitch.
So while they might lose customers like you, there is clearly ridiculously large piles of money up for grabs if they diversify their products, rather than remain specialized. And, of course, any sufficiently good specialist is at risk of being acquired by one of these behemoth generalists.
Although there is a good % of the market that works in a similar way to your business, it is worth noting that when a deal is exceptionally good, executives will create exceptions. MS Suite and some of its satellite offerings (PowerBI,Sharepoint,etc) is a good example of an exception.
There are also cases where a company will pick multiple vendors in an attempt to de-risk and/or for negotiating tactics. If you are fully dependent on a single vendor, the cost of migrating tends to skyrocket and the negotiating power moves towards the vendor.
> If you are fully dependent on a single vendor, the cost of migrating tends to skyrocket and the negotiating power moves towards the vendor.
This is particularly important with SaaS, as you lack the leverage to walk away. If you're fighting with a vendor, you had best resolve it by your contract date, as they will happily shut you off and wait for you grovel (and pay).
Yes, that seems more like Walmart/Target not wanting to support their main competitor in the retail space (Amazon) even though it's a different division.
I remember the wave of BI vendor acquisitions in 2007-2008 when IBM acquired Cognos and SAP acquired BusinessObjects. Now, 12 years later the wave repeats. Google is buying Looker. Salesforce is buying Tableau, and probably Qlik will be acquired within the next 1-2 years too.
Qlik was sold to Thoma Bravo, a private equity firm, about two years ago for $3B.
Not sure if that's a relative bargain compared to this deal or if it makes the $15.7B look totally unrealistic. As of a few years ago the total revenue of the two firms wasn't that different.
I've heard that Thoma Bravo specializes on re-selling assets. If that's true then the $15bln sticker on Tableau should only justify and boost their intention to re-sell Qlik.
My interpretation of the rise of tools like Looker & Tableau is that they happened in part because of that '07-'08 acquisition spree. Neither IBM not SAP capitalized on their investments, and instead their products mostly stagnated. Cognos 11 for example looks more modern bit really isn't much more than a re-skin of a product that hasn't changed much since the release of 8.x. At some point IBM had a Watson-branded product that was supposed to be a Tableau competitor, but it never really materialized and was near completely divorced from Cognos.
Indeed, it's cyclical. About 7 years ago there was a social media platform buying spree. Salesforce bought Radian6 and Buddy Media (for ~$700M). Oracle bought Virtue for $300M. Google made its own acquisitions in the space (Wildfire).
But now, Salesforce now has a bigger war chest to play with.
Yeah, Salesforce presentation of pricing really can feel like a bait-and-switch. Advertised as a pay per-user per-month, but then to actually make the system work you get all of these add-on fees, like for storage, or for API calls to perform data integration, etc.
I'm the CPO at a SaaS data catalog and analysis hub company called data.world - I was at the Snowflake Summit when the Looker acquisition by Google was announced - the guys at their booth seemed pretty happy ;)
With Google snapping up Looker ($2.6B) for Google Cloud, Salesforce's much bigger purchase of Tableau is a clear sign that the big guys see buying BI tools is a good way to expand the reach of their offerings into more of the business. We talk to companies every day that have made massive investments in data warehouses, viz tools, and high-paid data scientists, but they still aren't agile enough because they can't tie it all together so their people can find and use the right data when they need it. I read an article once about the failure of self serve BI and the reality is that you just end up creating more sprawl. People need tools to reduce the clutter and sprawl and stop the endless chain of emails trying to figure out what table to look at or query to use or if your source is still updating.
We built our data catalog and analysis hub for exactly this, and it's extremely validating to see the big guys like Salesforce and Google investing in expanding the user base of big data tools and I really hope we can be part of the solution of sorting it all out!
I disagree with you! Looker is more advanced, and that does not mean it is better. Tableau’s tools were MUCH easier for me to make ad hoc analyses with, as well as to create dashboard with. Yes, there were some transformations that I couldn’t apply and had to go to custom SQL or ETL for, but the total time to finish my analyses and produce high-quality live dashboards was much less than with Looker.
(I don’t know why you are being downvoted though, you’re entitled to an opinion same as me)
Oh hey, maybe this will mean somebody at Tableau has to start to give a damn about enterprise features, such as a way to do product activation and registration that doesn't fail completely in non-persistent virtual desktop environments.
Or Tableau can continue to pretend that this isn't a real issue and stonewall customers and partners alike.
I think the actual visualisation part is neat, and better than many competitors, but many of the server-side parts are various levels of disastrous (as is their support), and their "data preparation" tool needs some serious improvements to be borderline useable.
15+ billion seems like a lot to me given how Tableau interacts with customers and partners alike, especially seeing how they are activelly alienating existing enterprise customers, all in favour of new sales, but perhaps something will change for the better here.
Yeah as the defacto Tableau Server administrator in my org, I am not happy with Tableau. Despite it's power and ability to let less-technical people generate insights from data, the administration experience as a whole is a total nightmare; purchasing/applying licensing to all installations is a disaster. Maintenance upgrades and backup procedures are also sort of wonky. For the data visualization piece, I personally feel like the software is difficult to use because it uses custom "branded" terminology for many things, so the knowledge you gain using Tableau can't often help you outside of Tableau... and then they place many features in locations that are not intuitive and thus difficult to find without documentation or prior experience.
We're a Microsoft-heavy shop, and I've been trying to get them to move to Power Bi simply because it's far more fully featured, easier to use if you're familiar with the "Windows way" of working, and has streamlined administration/installation/licensing/configuration in Windows environments.
I think Tableau Server works better on Linux, where upgrades etc. are also somewhat less painful and require less downtime.
That said, it baffles me why I have to restart Tableau Server 3 or 4 times during installation, and why I have to restart it for trivial changes more generally. For a piece of software that specifically ships with a cluster controller and full-blown zookeeper, somehow their engineers (or "engineers", as I sometimes get the impression) manage to make things that should be trivially solvable with reloads, partial restarts or spawning new workers (e.g. SSL certificates for the built-in Apache webserver) require a complete restart of the whole node.
edit: Regarding Power BI -- I feel that Tableau Server is (for better or worse) one of the killer features for many enterprise customers, because it means all of your data can remain within your own infrastructure and does not have to rely on external cloud providers. If that is not a requirement in your organisation, Power BI might make sense depending on your overall IT landscape, as well as your users' specific needs. On the other hand, if your organisation requires hosting things yourself, I guess it doesn't matter how miserable the experience is for you as an administrator. That's basically Tableau Server in a nutshell.
this is a vote of no confidence for the Salesforce Einstein Analytics ( formally Wave analytics ) product. I suspect over the next couple years you'll see Einstein Analytics phased out in favor of Tableau with a SF stylesheet applied and framed inside Salesforce.
On the other hand, the IOT team at SF probably loves this. I got to spend a few days with the main engineers discussing their "Thunder" architecture a couple years ago. Under the covers it's awesome (great integration of multiple open source technologies) and the "IOT Cloud" UI isn't bad either but they didn't have an answer for data visualization on the scale they built for.
on a tangent, while getting into SF's IOT product they were talking about how it was so easy business analyst could set it up (famous last words right?). It's pretty easy to make a mistake and create a billion SF cases ( case is like a trouble-ticket in SF's Service product) in the span of "click -> "oops let me undo that" -> click" hah
I thought the engineering behind some of that stuff was pretty cool too. Unfortunately, a system that prides itself on handling millions of records a second AND has no version control AND is sold as "no technical skills needed" is just too scary for me to recommend to anyone.
Hey! Glad you enjoyed what the IOT team was doing! I didn't think many people knew about us!
I was actually a part of that team and left Salesforce pretty recently. I think i can safely state that everyone involved in building the tech for that IOT product was pretty proud of what we built.
That being said, the team that built it 2-3 years back and the current team are fundamentally different. The department was thoroughly axed back around November 2017, and multiple teams, including my old one, were spun out into different other projects. It's a shame really, that project was what lured me into joining a behemoth like Salesforce in the first place.
Wow we probably crossed paths. At the time, a CTA buddy of mine was starting the North America IOT Cloud practice at Accenture and was bringing me along for the ride. He had a baby and eventually left to run a program at a client (9 to 5 no travel). Without him, I bailed soon after for a higher up role at a startup. I would still love to get back in that scene though, this acquisition makes it that much more interesting.
The move to buy Tableau likely comes from an interest in enhancing cross-compatibility between Salesforce+Tableau, the ability to provide a more robust service offering that can compete with Microsoft (e.g. If you buy our CRM solution, we will give you Tableau for a 25% discount) and a concern that another big player would have come in and taken Tableau.
I’ve used PowerBI but it feels extremely limiting if you have a coder background. Don’t know a single person who likes the BI tools that exists today and I’ve worked with a lot of BI people. I’m convinced BI people becomes coders solely because of the bad tools that exists today. Intelligent autocompletion, git and strong types? No! But here, have some drag and drop!
PowerBI is not for developer. It's for Data Scientist/Business Analyst.
Someone will come and build the dataset (or known as model) and the BA/DS can start building visualization that make sense to tell story about the company (or to tell the story about something).
When we started Graphistry, our view was, "skate to where the puck is going", so while we saw Looker and Periscope burning sales/marketing/dev $$$, the result felt 95% similar to Tabluea Public. Instead, we've focused on figuring out how to harness next gen -- GPUs, how to expose ML and automation, etc. While I still think we are right long-term, and that the cloud co's will be paying top dollar here for the next 5 years.. Short-term, I didn't expect this much market hunger for old-school.
I'm responsible for external salesforce integrations at my company, this is a really critical acquisition. Salesforce feature categories are surprisingly easy to compete with (be it LeadSourcing, Pipeline Management, 3rd Party App distribution, Licensing, Performance, Design, etc.) because they do a relatively average job at each of them. There are dozens of competitors springing up, many of whom target one of these categories as a key selling point.
Now if you go into a fortune 500 sales meeting to pitch how much better the analytics stack of your CRM is, they will come back saying that Salesforce is clearly investing to be superior in that category for the long haul.
Honestly, while doing some business analyst job for a company, the higher up keep on bringing Salesforce as something we should consider for a B2C project. Their marketing force is something to recon - and the lack of basic knowledge about what they wanted to put on the market was frightening.
So, they’re not the best? People don’t seem to think clearly about this :(
I would NOT recommend it unless you have deep pockets for syncing to a 3rd party database like Amazon RDS or Heroku connect. Mostly because your database schema will go to shit real fast either because business users will modify it arbitrarily or because you'll struggle to understand the performance implications / indexing requirements of various Salesforce tooling.
Sometimes its easier to buy the second best at everything, (caught Microsoft) and benefit from everything supposedly working together, and a clear cohesive overall roadmap, than trying to stitch 200 individual products together, make them all pass data to each other naively, and hope that their feature changes and roadmaps stay aligned.
Oh yes ^^^ this. Where I work we recently upgraded from a single legacy product to another ERP, only for many secondary functions we went "best of breed" to additional SaaS products. It has had, literally, an exponential increase in the necessary resources to make them all work together. Keep in mind, the ERP we upgraded to also had modules for these secondary functions, but it was a case of Perfect-Is-The-Enemy-of-Good, and the sum of all parts is now much less than the mediocre Good we would have had with a single ecosystem.
I used to teach Tableau too and while you're right, the tool still becomes useless after graduation because of the ~$1000/yr price tag.
From there, tableau becomes a "learn it just to get hired at a company that might use it" tactic, which is a valid job hunting strategy but is not something useful to learn in a data viz class, especially since there are other data viz tools the students can learn and keep using after graduation.
Are you teaching how to use Tableau (the product) or Data science/BI? Because I don't think you should be learning how to use a specific software product at school unless it's vocational.
That said, I don't see why any student couldn't continue using the free Public version.
>I stopped teaching Tableau because of the licensing costs to my students.
Weren't you using offer of free licenses for teaching environments? Those are full versions, albeit they do not include Server, but that's not really necessary to learn how to make solid BI dashboards.
I'm still really interested to see if there's any long term blow back for Salesforce from the whole Camping World saga.
Even for people who agree on principle, as a business owner in general I'd be very wary of using software from any 3rd party who's willing to try to influence how I run my own business. It makes me wonder about all of the other companies under that SF umbrella too.
> I never really understood tableu, it just makes fancy graphs from your data doesn't it? What's new about it?
It's really easy for business users to point it at a database and get started on their own, exploring the data. It feels like it has a much lower barrier to entry than many other reporting tools.
And once you win the hearts of executives, that's kind of the end of that discussion. It's a really sticky product.
From personal experience, Tableu has saved my team countless hours adding reporting features to our products just because some middle manager on another team got a hair in their butt, so I can see the value.
I once accidentally spawned a five-person department at a Fortune 50 company because I threw together a Django app that integrated with Active Directory and dynamically displayed a few reports that were relevant to the logged-in user's role in the organization.
I was in Corporate Strategy, and one of the C-level execs saw it. They realized that I could add new reports in a few days by myself instead of going through IT, and within a year they'd hired a new manager to run my little group and three others developers - who weren't technically developers, because we weren't part of IT.
For another ~3 years, the company's primary reporting tool for front-line operations employees lived on a repurposed desktop being used as a server that was plugged in under my desk. We did get a nice, beefy SQL Server and a decent UPS, so it wasn't a complete shoestring project.
I left shortly after IT discovered they'd been cut out of the loop on a critical business process and demanded that the whole system be rewritten in C#.
"I left shortly after IT discovered they'd been cut out of the loop on a critical business process and demanded that the whole system be rewritten in C#."
been there, done that. you have to get them on the phone early and do the whole "oh you're so great" song and dance to prevent ego bruising.
when i scope projects for clients if I have to engage with their IT department I add 6-8 weeks of implementation minimum. Anything from getting a CSR signed to SSO integration is always an excruciating experience. Going over the SOW and hitting on that item always results in a slow, exhausted, but accepting sigh from the client.
I think you might be surprised at how many people do know what a join is, they may call it another name, but it's not a hard concept to understand. It's a very common thing I've seen non-technical people do and use.
Yeah, one of my old coworkers was ecstatic when he learned about joins. He'd been downloading two tables and then using vlookup to join them in Excel. Using actual joins sped up his workflow a lot.
Thats why their non-technical user abstraction is a worthwhile investment for businesses who want their devs to be doing real work instead of pandering to the latest whims of a product director / marketing exec
At my last job I had a nontechnical boss, and I would have been ecstatic if he had a) enough of a clue to try screwing around with SQL, and b) the wisdom and humility to ask for help.
Yeah, I occasionally have to visit folks and walk them through extremely basic things like sorting data in Excel and creating pivot tables (very much not my job, I'm general purpose BI development mixed with statistical analysis & ML) but I never begrudge those meetings because it means the user is trying to become empowered to work more independently. No matter the specific skill level, that's something I can get behind.
At my place, somebody a while back drank the Tableau koolaid and quickly designed some nice looking graphs and sold the commercial team on its value, and the president signed a check for a license.
The software team was then tasked with putting the fancy graphs that guy did into our web application with the following constraints:
- the Tableau server can't be exposed on the public internet
- our UI can't indicate that it's using Tableau anywhere (i.e., use an iframe or something)
This turns out to be tricky. Tableau doesn't really like to be embedded in 3rd party applications, it leaks information about itself in a number of places. It requires that every person looking at the graph be a user according to Tableau's definition of user. Synchronizing authentication to Tableau server and workbook authorization gets tricky.
The next task coming up is that they want users to be able to ad-hoc schedule an email to themselves with the fancy graphs attached as a pdf, so we've got that to look forward to.
I worked on a data dashboard site that embedded Tableau workbooks. I found their JS API to be decent for embedding workbooks[0]. I used what Tableau calls "Trusted Authentication" to handle the authentication to Tableau Server[1]. It's harder if they are using Tableau Online.
>The next task coming up is that they want users to be able to ad-hoc schedule an email to themselves with the fancy graphs attached as a pdf, so we've got that to look forward to.
This is the endgame of every OLAP tool. But execs really love this feature.
1) An effective tool for people to explore data (with a relatively low barrier to entry - some training required).
2) An effective dashboard authoring tool (i.e. to make small specialized data reporting apps) which are simple enough to be used by anyone without training. These dashboards typically give some sort of situational-awareness for key performance indicators (KPI) such as sales, inventory, etc and are highly specialized for a specific use or role.
Imagine how useless an airline website would be if it were non interactive reports of all the data for the day loaded onto one page. Good dashboards are interactive, not passive reports.
>This change in approach could be equated to going to individual airline websites to check routes, dates, times and fares of flights as opposed to just going to a website like Orbitz or Travelocity – punching in where I want to go and when, and it pulls in a report of all the flights that meet my criteria. I can then narrow those results down by a number of criteria – time of day, number of stops, price, etc. It’s self-service reporting in the truest form.
Making fancy graphs beyond standard spreadsheet templates can become time consuming / labour intensive very quickly. More enlightened companies will not hesitate ot pay for Tableau if it relieves their 6-figure salary data scientists from reporting/dashboarding work.
I think PowerBI compares with Tableau quite favorably. For the most part PowerBI is a clone of Tableau, but it goes farther and adds in much more powerful scripting(M, Python, R), a library of custom visuals, Power Query which is a nice GUI for data transformation, and deep formula language (DAX). I think it's easy for an Excel or Tableau user to pick up in a day, but it scales much more as you have more data analysis expertise.
Also MS has been improving it like crazy over the past few years. They clearly want to maintain their dominance in the data analysis space, but see the winds shifting away from Excel and are not waiting to become irrelevant.
The model isn't that a business user would use Python or R. Rather, a data scientist shares live-computed analytics to business users through a report / dashboard.
Microsoft has PowerBI which is already almost magic for many business users with it's ease of connectability with different databases and SaaS services. Transformations, parsing, mixing data from different sources and reporting capabilities are also easy to start with and quite powerful in the right hands. It's not integrated into Excel directly but sits quite well imho in the MS ecosystem.
PowerBI wants to be Tableau based on the number of times Microsoft sales has demoed and whatnot.
But I haven’t met databiz people who switched from tableau to PBI. I’m not sure why as they look pretty similar on paper.
I think Tableau’s mental model is about exporting and storytelling while PBI is about reporting.
Personally, I find it harder to work with PBI because it’s the only license model more confusing that Tableau. And there’s no Tableau Public equivalent.
This - we had power pivot in excel - then for some licensing reasons we didn’t. I know how to buy excel and firms buy it blindly - tie power bi into that sales channel
I'm not the OP but having dealt with QuickSight it really feels super flaky and difficult to create graphs with. I can't remember the exact problems my co-worker had problems with it, but if I recall correctly one of the problems was that QuickSight expected all JSON data keys in S3 to be in the exact same order in every file thus making changing the schema a very huge pain in the butt. Also the UI itself is not good and creating eg running monthly average was very non-intuitive. A lot of small stuff that makes you not wanting to deal with it if possible. But it's cheap and has good AWS integration so I guess it has its purposes.
Coming from an enterprise background in BI using traditional tools like Cognos and Business objects, Tableau is head an shoulders above these. It's not only significantly easier to build complex interactive dashboards with Tableau, Tableau enables a level of rapid data exploration that simply isn't possible in the traditional tools. In those, you pretty much needed to know exactly what you wanted to build, making it difficult to find new insights.
It's like Access for business people and in many cases gets driven by the business, not IT. In many cases, it cuts out the IT SMEs that govern access to something like business objects.
They also do a good job at seeding it in visible places. Alot of newspapers are using Tableau for infographics, etc.
The primary value is making interactive data visualizations with minimal effort. I can go from nothing to having something in the user's hands in less than a day, complete with things like tooltips, dynamic filters, parameters, etc.
I love Redash, but Tableau has a much larger set of features. Redash is pretty good at queries and basic charting. Tableau is pretty good at queries and basic charting, but also offers a bunch of extra visualization types, predictive analysis, native cohort tools, etc, etc.
(That being said, I think a lot of teams are quick to reach for heavyweight tools like Tableau when Redash would be perfectly good for what they need at far less cost)
I think that many folks don't remember or know about how the word "BI" [business intelligence] was born.
Back in the day corporations paid programmers and consulting companies millions of dollars for the purpose of building out applications with databases upon which to digitize their business processes. This was a great for a couple decades, up until some time in the 90's.
It was at that point that expectations had caught up with digitization and having your battle-axes in accounting/order-entry/supply-chain do their keyboard magic on green-screens just wasn't cutting it. To actually understand what was going on, managers had to ask for specific reports from these people and they were just not in a position to do the reports as well has get their own ever-increasing-shit done. Let alone whatever the hell sales was doing.
For a control-freak in management this just isn't acceptable. They started to find ways to "serve themselves". Eventually someone had the idea to hook the enterprise databases into their own zany spreadsheets. It was amazing. There was such a fountain of knowledge and insight that it made the people who could do that seem like relative geniuses. Almost "intelligent" in their business jobs. Thus, "Business Intelligence" was born.
"Business Intelligence" is the practice of making badly-designed opaque shit from brutal, inscrutable business applications... visible.
Only now are makers of enterprise applications starting to get it (or pretend to get it).
I was recruited for one of the first engineering positions at Tableau a very long time ago, but I turned it down as it just wasn't interesting enough. I think I was targetted in part because I was a SIGGRAPH presenter/author as one of the co-founders, Pat Hanrahan, was big into SIGGRAPH. Oh well.
I'm shocked to see this not being a better day for Elastic. While still quite narrow with ELK stack, the elasticsearch backend is something that many more companies can make useful quickly given how non-finicky it is.
Very happy for the Tableau folks, they are great to work with!
My firm uses Informatica's IICS to output all of the datasets for Tableau to visualize, its not as exciting as the hundreds of frameworks and languages that do the same but it works reliably and I don't have to hire expensive data scientists to get the job done or switch tech every few years to whatever is the flavor of the month.
We take the 80/20 approach that most questions the business asks can be answered with Informatica. The last 20% we save for our developers and the odd data scientist to help us with.
I really really hope this makes Salesforce more like Tableau and not the other way around. Their UX and workflows are atrocious (not that their main competitors' aren't).
At its core, Tableau is an implementation of a really elegant formalism to specify visualizations, and it's much more organized and declarative than something like Grammar of Graphics.
Just for the comparison, salesforce's revenue is ~$13B. At current profit rate, ( including that of tableau ), they can generate $15B in about 30 year, may be 20 year given growth and inflation. I run a small business, I don't think I would be able to generate equivalent of deal, nor my banker/investor allowed such a investment, no matter what growth would look like.
Ultimately my take is, small business usually take growth from someone else most of the time ( i.e. new restaurant steal customer from another old customer ), instead of generating brand new growth, while bigger business are actually creating - or - consolidating whole industry for growth in a society-positive way in the long run. ( like uber ).
Still somewhere I would wish I can replicate such method of growth for small businesses.
Wow. I think that’s a very insightful perspective, but come away with a very different perspective:
Big companies play by very different rules than small ones, especially big tech companies right now, and I don’t see consolidation being (broadly) socially positive at all really.
Last week Looker, today Tableau, next week Qlik or Alteryx?
I wonder how this will integrate with Einstein and other AI products salesforce already has. Pardot is the first to come to mind. They already own Heroku, Mulesoft, and Quip.
Exciting times to be a data geek. Hopefully this adds more money to the AI race.
On a sidenote, at what point do we stop taking Gartner seriously? Recent report had MicroStrategy (complete dinosaur of an application) ranked top on almost every category. It's so blatant.
Looker, Periscope, and Tableau have all been acquired or merged in the last six weeks. Periscope merged with another BI start up Sisense. Looker and Tableau were part of a multibillion dollar acquisitions. It's an interesting time for this space.
I think the value will be created from the data itself, less focus on data-processing skills. Of course, modeling skills will be still important at future, but data-processing would rather become much more easier. That's why self-bi tools like Tableau, Elastic search are becoming more and more popular.
Salesforce appears increasingly worried about being pigeonholed as a CRM product and wants to ride the data wave and be seen as a data product.
I’m not sure this acquisition makes a lot of sense though. The Tableau fad seems to be fading as people realize that while it’s a nice glorified Excel it doesn’t really solve the data issues most companies have. It seems like a very high price for a product that may be past it’s peak.
That said, we’ll done to the Tableau team for having gone public and then selling the company again at a nice bump.
The bloom may be off the rose with Tableau, but I'm not sure I understand the "glorified Excel" comment. Sure, Excel has charts and pivot tables that might have something comparable in any single visualization in Tableau, but its inherently non-relational and single data source, where Tableau is made to mash up data from multiple sources for exploration and creation of interactive dashboards, something Excel doesn't really do very well, if at all.
I can hope they leave them alone, but that's a fools wish. Not looking forward to them stripping away feature after feature, and charging exorbitant per-user prices to get them back.
Anyone know how something like Tableau compares to something like Palantir? I don't have any experience with either but I've heard that Palantir is valued quite high as well.
That's a pretty steep price to pay for a single-digit % increase in revenue. I view this more as filling a strategic hole. BI & Analytics from Salesforce are awful.
Can you elaborate on the advantages of Periscope? I've used Tableau, but not Periscope. Browsing their site & marketing materials, it's not really clear to me how they're much different from Tableau.
I think the development of Python and Jupyter and other less known things like Vega are much more interesting. Python is today the only "glue code" that puts all of it together, from data to insights.