Instead of complaining about the structure or motivation of business rules, excellent developers ask different questions:
- How can I design my architecture to accommodate just about anything my customer throws at me?
- How can I use all the tools in my tool box (including CS and math concepts) to clarify the apparently nebulous?
- How can I use things like Boolean algebra, parameter driven program logic, and data base normalization to provide structure to that which appears to be unstructured?
- How can I use the customer's seemingly unreasonable demands to differentiate myself from mortal programmers?
- What do I need to become in order to say "Yes" to my customer whenever I want?
- How can I use all of the above to actually encourage my customer to attempt the "undoable" in order to differentiate himself from his competitors?
Having spent the last 9 1/2 years working on systems for property and casualty insurers, I have to say that I don't really agree with everything.
I generally agree with the point about designing your architecture to accommodate whatever gets thrown at you, but doing that is hard. Immensely hard. Nothing is concrete, everything is some extensible meta-problem, where instead of writing code to do X, you have to think "how can I make it so my customers can do X, but can also change it to do Y instead." And of course, they want it to be easy to make it do Y instead. Do you just make them write custom code? Do you enumerate all possible options in a declarative fashion and let them choose between them? It's never easy.
The bits about finding structure in what's unstructured, though, I find to often be exactly the wrong mentality, though it's usually the mindset all engineers start with. The best approach in my experience is usually to just embrace the fact that everything is arbitrary, and recognize that you're just going to be writing a lot of if statements (or giving your customers ways to write if statements); otherwise, you'll inevitably A) run into some situation your nicely-structured algorithms can't handle and B) be endlessly frustrated by the exceptions. The processes being modeled by enterprise software are fundamentally irrational, illogical, inconsistent, and arbitrary. Accepting that, and building systems that can handle that, are hard things for engineers to do, since the tendency is to always try to find patterns and order and simpler, more general algorithms.
Not all problems in enterprise software fall into that category, so it's critical to be able to identify which sort of problem you have: is it one that's structured and makes sense? Then write code accordingly. Is it modeling some pre-existing human process? Then expect that the crazy exceptions you've heard about so far only comprise 1% of all the crazy exceptions out there, that you don't even know how to categorize what those exceptions could be, that there's no real underlying order, and that your job is to write the system in such a way that it can handle the fact that the process is arbitrary.
My first job out of college was with a consultancy that specialized in payroll integration projects. We were a partner with ADP (largest payroll provider in the country), and typically they sent us the clients with the trickier payroll processes, so know I'm speaking from experience.
Long story short, I agree with what you're saying, but you make it sound like it's really simple.
It's not.
Creating a single payroll solution to handle one company is very easy. Creating a single payroll solution to handle many companies is also quite doable.
But once you get to the scale of ADP, it's impossible to handle everyone elegantly. Between union contracts and non-standard practices, there are (what felt like) an infinite number of things a customer can throw at you, with one rule wackier than the next.
It's damn hard to write a generic payroll solution that handles most companies. Damn hard.
Not thinking about it is not the problem. A lack of domain knowledge is. If you don't know how payrolling actually works in practice, no amount of thinking will make you come up with the right solution. The real pitfall here, and the downfall of many startups, is thinking you can understand the domain merely by thinking about it. Engineers can usually think pretty well and can solve many problems in that way. It's crucial to realize not everything can be solved by thinking about it.
I seems like the general system is something like:
A) A set of tools to talk with external systems
B) A set of tools for manipulating the internal representation of payroll data
C) A GUI for interacting with / managing the system.
When their internal process is simple managing the payroll system should be simple. When their internal process is complex you get fat consulting checks. Which brings up a great point, it seems like large companies that simplify their internal systems could avoid writing a lot of consulting checks, so I suspect they must be out there and you just don't hear about.
1) There is "one" way of representing payroll data. There's not. When you get a none trivial amount of clients, all their edge cases put together destroy any ability to fit the data into one conceptual model elegantly.
2) Companies have control over payroll rules. Large companies often can't simplify their systems due to contracts or the subsequent employee mutiny. Unless you want to get sued, lose the lawsuit, and pay out the nose (which I've seen happen), you can't just change how you do payroll on a dime.
Not to mention, there's a ton of disfunction too. Payroll managers aren't always the brightest tools in the shed. One project I worked on lead to the payroll manager getting fired over "bad stuff" I uncovered. And then as I uncovered more stuff his replacement was let go too.
...payroll can be a total clusterfuck. Avoid at all cost.
1) The only conceptual model I think needs to apply is the how things are taxed. Granted some organizations have edge cases in how they pay taxes (or how they try to hide things) but I think innate complexity dominates unique edge cases when it comes to the tax code.
2) I would argue that much of this complexity shows up because many companies negotiate while ignoring the cost of adding complexity. Company wants to minimize cost, workers want to maximize benefits, complexity is a dead weight that benefits nobody. Once things have been negotiated it's hard to change, but during negotiation they is no need to end up with yet another edge case.
PS: I don't think organizations end up spending 100+ million on their payroll systems because it's easy. I just think being proactive lets some companies minimize those costs. EX: Is Google's payroll system ugly?
I totally agree. but, It is not impossible. The problem with Payroll is that it is like Security. When everything is working, no one notices its presence. Any issues are incredibly important to fix.
That said, there are ways to develop payroll software that keeps extension possibilities and accommodate the variations without disrupting the engine.
I came here to say the same thing, and you'd already said it.
I too love problems like this. A properly designed system like this one will account for every possible oddball situation you can come up with -- and if it can't support an especially weird solution out-of-the-box, it will be easily extensible such that it can.
Too many people code to the goal and don't try to model the underlying logical concepts.
For some value of excellent. Simple, general rules, that are well understood, are better than complexity, even complexity your excellent developer can manage.
How about a developer, who says no to their customer, and comes up with something simpler, and better.
> How can I design my architecture to accommodate just about anything my customer throws at me?
Poor attempts at this will result in very complex systems with unnecessarily many layers of indirection which still can't accommodate to the most peculiar business requirements thrown at them. It's like having the worst of both worlds, overengineering and dirty hacks.
Pretty much all "enterprise software" is fundamentally the art of taking totally irrational, human-defined, fuzzy, incompletely-specified pre-existing processes that have been built up over decades or centuries, and turning them into an automated system that functions more or less identically. It's fundamentally different than doing something like writing a word processor or e-mail client which, while it has its own complications, doesn't suffer from having to exactly model some pre-existing, half-specified, inconsistent process that was developed by humans long before the advent of software.
I often wish there was some way to inject some order back into the real world systems. So many of the techniques and processes that developers take for granted seem entirely absent from our regulatory and administrative law systems:
There are many ways to bring some order back into the world, as you say, but it can be a struggle. In my experience, business leaders will support simpler systems and processes if (and only if!) the benefits clearly outweigh the costs.
As an IT professional, framing the right perspective is essential to getting acceptance for better systems. Presenting a more powerful, but simpler, system as "losing some rarely used and inefficient features" is a sure way to get that project rejected. Management will only hear the "losing" part as a loss in total value and therefore worse than the current solution.
Personally, I like to present formalized system upgrades as the price of growth and future growth potential. Most business leaders can wrap their heads around the concept of "we need to focus and streamline in order to reach the next level", regardless of technical experience.
I guess I wasn't clear. I realize that businesses need to survive in the eco-system created by crazy laws and ill-thought out regulations and contracts.
I was trying to suggest that the legal and regulatory framework is what needs to be fixed, not the businesses.
Ah, look at that, I totally misread where you were going! ;-) In my experience, regulations and crazy laws aren't really a competitive advantage or disadvantage because all of the market players are playing by similar rules. I'm not saying that there's no negative effect to bad regs and laws, but the costs are shared fairly equally.
I've never worked at a business that was cut off at the knees by new laws or regs. I'm sure it's happened, but I'm not convinced it's common. What I have witnessed though, many times at many employers, is pure self-inflicted complexity. The did-they-even-consider-the-cost kind of complexity that is nightmarish to implement and hampers future changes.
As an IT pro I can't do much about the laws and regulations, but I can advise and design business systems in ways to avoid complexity traps and help accommodate more sane business processes.
>As an IT pro I can't do much about the laws and regulations, but I can advise and design business systems in ways to avoid complexity traps and help accommodate more sane business processes.
This, I think, is a very healthy attitude; Focus on what you can change rather than railing against things that won't.
This is where entrepreneurship comes into play (at least in the long run). Many older enterprises become uncompetitive because of the chaotic absurdity of their systems. This allows upstart new entrants or innovative competitors to eat their lunch.
There was once a programmer who was attached to the court of the warlord of Wu. The warlord asked the programmer: ``Which is easier to design: an accounting package or an operating system?''
``An operating system,'' replied the programmer.
The warlord uttered an exclamation of disbelief. ``Surely an accounting package is trivial next to the complexity of an operating system,'' he said.
``Not so,'' said the programmer, ``when designing an accounting package, the programmer operates as a mediator between people having different ideas: how it must operate, how its reports must appear, and how it must conform to the tax laws. By contrast, an operating system is not limited by outside appearances. When designing an operating system, the programmer seeks the simplest harmony between machine and ideas. This is why an operating system is easier to design.''
The warlord of Wu nodded and smiled. ``That is all good and well, but which is easier to debug?''
He devoted an entire session to the French Railroad payroll program. The French Rail- road had gone on an economy program and reduced their number of employees from 540,000 to 360,000. So that was the size of the payroll that was run on the computer in Paris. That payroll must be the most difficult payroll in the world to run. The size of the payroll is only the tip of the iceberg. An engineer’s (i.e. a train driver) salary is a function of the amount of power that he uses. So if he is on an uphill run, he makes more money than on a downhill run.
The employees live in government housing and their rent is deducted from their pay. People with a lot of dependents get more money than people with a few dependents. A vet- eran gets more money for each medal that he won in the war. The amount of money depends on the medal. A Purple Heart is worth more than a Sharpshooters medal.
Another one I saw recently: "Oh, you've worked from that alternate location for too many days this year, so your travel expenses are no longer a business expense. The company will still pay them, but now they are taxable benefits. So the company will actually pay you extra in payroll to offset the taxes you will now have to pay."
Healthcare software has the same kinds of problems. New regulations, rules, and ways that each party in the system attempts to skirt the rules bleed into the code bases of every system.
Thats why a lot of the technology in healthcare is so far behind. One place I worked at still makes $38 mil/year on a system written in C++ and ASP. Replacing it with a modern stack has been on going process for more than 3 years with no end in site. In the mean time, new installations are being brought up with a whole new set of business rules being added that will just have to get ported later.
For national companies, state level reporting requirements are a huge burden. For instance, payment made to doctors by healthcare companies. You just hope that they pick fairly common requirements, instead of each state reinventing the wheel.
I just built an internal payroll system for my company. It's been a huge pain because it involves both parts of the business I generally try to avoid - legal, and dealing directly with money.
Oh, a big item missing from the list is dealing internationally. If you think navigating all the state and local laws are hard wait till you look at another country :)
"The debators of the rules are more interested in satisfying the group than in definition or algorithmic simplicity or purity of concept."
This sentence is incredibly useful for understanding why modelling a process hammered out by more than three people will always be more complicated than you think it is.
Actually, Payroll systems can be structured. In the end, there are only 3 things that you need to provide:
- compensation (money employee makes)
- deduction (money employee gives for some service)
- taxes (employee owes the government)
Everything that you do is toward satisfying those requirements. It is hard because each of these pieces are influenced by human interaction. But, that does not mean there are no rules of engagement. e.g. If you have union dues to be deducted, it is a formula to compute it against the total wage you make (or it is a flat deduction amount). If you are paying towards 401k, it is a percentage that you put in towards that.
If you imagine the problem as something to solve in a general sense, you can narrow the problems to fit into models. But, if you treat every requirement as a modification to an excel macro (:-D), then you are in trouble.
They get nasty. You cannot underestimate the complexity that collective bargaining agreements bring.
New York City, as an example, has something like 15,000 variations for payroll categories. Plus garnishments, domestic relations orders, etc.
How do you compute the extra boot benefit for a fireman who has worked more than 526 hours in a 13 month period?
Or how to you compute the salary of a garbageman who is paid by the ton of garbage hauled, instead of the hour. Except for when they haul ashes, in which case the tonnage is adjusted, or on Sundays, where they get overtime as well. What are the business rules for an ash hauling garbage guy earning overtime on Sunday, which is also a holiday?
The NYC consolidated payroll system, "CityTime"... has cost over $700M since 1996 to build. (That included some serious fraud.) Fraud aside, Accenture was raping & pillaging too -- charging $400/hr for college grad coders.
I am really not underestimating the effort. But there are multiple pieces to what you just talked about. There are complexities in time measurement (or computing how much work was done) and in payroll calculation.
When systems of this sort are built, it is usually a combination of Timekeeping (how to measure time spent on the job) and payroll (how to pay for that time). Each section is a big (i.e. millions of line of code and many engineers) worth of effort. All that said, depending on how systems are built, you can
a) Create a base system that solves core issues
b) Accomodate as much configuration towards that system to allow computation to occur differently based on what the employees do
c) Allow customizations of computation when as required
d) Make most of the data that can be changed, to be changeable through design (data driven instead of code)
Of course, there will be customers who will want to behave differently because the moon is in a certain position in the system. But, that is when you need to know the domain enough to understand whether what they are asking for is because "that is how they have always done it" or because there is a valid use case that needs one of the b), c) or d) to be updated.
But most people have complicated personal banking set-ups with money flowing in and out to different places at different times. It works well because the individual has control. If you had to automatically pay for your shopping using some database magic every week, instead of just handing over a credit card it wouldn't work either.
Treat payroll like an individual bank account with facilities to make and receive payments automatically. Then let actual humans enter the information individually for each person. Add automation of this as an after thought once the problem is fully understood.
> If you have union dues to be deducted, it is a formula to compute it against the total wage you make (or it is a flat deduction amount).
Except when it isn't. For example, some folks have multiple roles, covered by different unions, with different rules. Those rules are applied to different parts of their "total wage". I suspect that some wages don't count for 401(k). And so on.
None of these things necessarily have simple definitions and they interact in complex ways. Any "general model" is likely to be wrong and seemingly minor changes can have huge consequences.
Actually, it is feasible. It is possible to do this still. Solve it with a model to start and chain the calculation events. You can re-iterate based on the different situations. The design, when thought through, can be accommodating such variations. It is definitely a lot of work (dont get me wrong). If this was not feasible, there would be no Payroll vendors. :)
> It is definitely a lot of work (dont get me wrong).
We agree.
My point is that the models are brittle because every entity (govt, unions, etc) has different definitions for every one of the "common" terms, can change their definitions, and the scope is very context and usage dependent.
If you imagine the problem as something to solve in a general sense, you can narrow the problems to fit into models. But, if you treat every requirement as a modification to an excel macro (:-D), then you are in trouble.
Wow, a new person's first comment hits the bulletin board!
Payroll systems should never be migrated wholesale. They should be scriptable, for instance pay masters should be able to script the computations to assist them.
Yes. Sometimes I think a good way to organize an enterprise system would be to make a trivial CRUD "layer" (no validations there) and then write a bunch of Prolog-clause-like rules. The rules would be the big and interesting part of the system, of course. The system would have to be intelligent enough so that, for performance reasons, after input, only the rules that could possibly affect the input would be run. But then in my modest experience these systems are so complex that sometimes for a simple new input you have to validate a whole bunch of things.
I currently develop health care related software here in Brazil. Total chaos. Once I was talking to a doctor, and he said engineers think mostly about the patterns but doctors think mostly about the exceptions. I think the same applies to business. The doctor in question of course liked our system but had a huge list of feature requests (to treat exceptions).
Totally. I used to write event handling rules for the Tivoli Enterprise Console, which used a Prolog dialect and a godawful DSL based on prolog.
I had a ton of fun doing it... basically you had to write rules to normalize SMTP traps from various vendors, and take action based on them. Network vendors did all sorts of bizarre things -- one ATM would periodically send a "Interface Down" event, with an attribute set that meant that the "Down" event actually meant up!
The hard part was actually doing anything with the events. The regime in charge of my division had a deep seated fear of automatic actions, even actions as simple as opening a trouble ticket.
C3 was indeed, but payroll systems have been written in pretty much every language under the sun, I'm guessing chollida1 was talking generalities (and the mess described in the article is the kind of stuff you find in every payroll system)
Typical noob mistake #73: trying to make a class out of every rule in a rules-based system.
I hate to be blunt (or worse, snarky) but seriously, everybody does this at least once: modeling and factoring out tons of bullshit when the user is really explaining an ad hoc rules system. Very painful.
At some point during analysis, you have to ask yourself "Do I really want to make the system more complex for each example of X the user gives me? Or just acknowledge that the user is free to come up with all kinds of crazy rules from now until the cows come home, and code that?" (and simplify)
Your reply reminds me of Zed Shaw's presentation 'The ACL is dead'. Basically he said that if the law rules that you have to handle are turing complete, you're probably better off with a turing complete language:
Of course, procedural code becomes spaghetti code quite easily, but one probably needs the same discipline when doing object oriented/relational modeling...
Payroll is a really weird beast. In some ways business apps can be a challenge, and in others a real pain. Payroll is definitely one of the harder business problems to make generic.
I've just come off a payroll project that has been 2 and half years in the making, and I can say first hand that all of these things happen. And more.
Acquisitions, benefits, and here in AU award wages for hourly employees make things even more fun. Different sites of the same company can have completely different set of rules and regulations that they adhere to, purely because the unions decided to negotiate differently for a different site.
Thankfully I've never worked with something this complicated, but it doesn't surprise me (though reading it did give me a good laugh). What I've learned is that when money is involved, any sense of logic and sanity immediately goes right out the window.
Instead of complaining about the structure or motivation of business rules, excellent developers ask different questions:
- How can I design my architecture to accommodate just about anything my customer throws at me?
- How can I use all the tools in my tool box (including CS and math concepts) to clarify the apparently nebulous?
- How can I use things like Boolean algebra, parameter driven program logic, and data base normalization to provide structure to that which appears to be unstructured?
- How can I use the customer's seemingly unreasonable demands to differentiate myself from mortal programmers?
- What do I need to become in order to say "Yes" to my customer whenever I want?
- How can I use all of the above to actually encourage my customer to attempt the "undoable" in order to differentiate himself from his competitors?