Hacker News new | past | comments | ask | show | jobs | submit login

One of the most under appreciated aspects of data is the processes and devices that create the data. I've found that most people think that all you have to do is link data x to y and voila get on your way. The successful integration of data across systems is more often a challenge of processes not meshing. I see very little in the discussion about how to overcome this process impedance mismatch. There needs to more agreement on where a process ends and its output.

On the technology side where the systems are complex and there is a need for further understanding I've seen some success with RDF where there is more of description with of relationships and one can form a topology. RDF though requires a big jump in knowledge and has only been worth it when there is no other way. Superficially it is simple, but it introduces a lot more questions at times that can lead to a lot of complexity.

I've seen standards generated in the utility space through CIM and IEC via extensive collaboration between vendors and the utilities, but the environment is very different with a lot less competition and narrow scope. This has also been achieved at great cost that most companies won't want to bear. Further, it has still been extremely hard as utilities have different network standards around the world. Look at things like IEC 61968-9 or CIM for transmission and distribution networks for examples. Green button is another example, but I would suggest it is easier as it is more of a final node problem and not right in the middle of a more complex process.

If the focus is more on FICO like situations, it is likely a lot easier as it is more of a MDM like view where there is a strong centre and much weaker nodes around that centre that get good value from the conformity. But it isn't clear if there are a lot of opportunities like that and that those likely don't need much of a strong standards base they just need to hit the right problem at the right time and define it as they like.

In dealing with marketplaces and being in the position as a supplier I've found that the supplier position is pretty weak and the marketplace does whatever they want. When using old standards like EDI they just put in very dirty (poorly processed) data and let the supplier just deal with it. There is a lot of preference to work with better marketplaces, but sales teams really don't care and the cycle goes on. The incentive models can be quite skewed.




+10 to this comment. linking data is simple in theory but extremely hard in practice.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: