A word of warning on adopting RPA

Preface

All technological choices are essentially tradeoffs between multiple factors and RPA is no exception. We have a natural and understandable bias to overemphasize the short term - being more immediate and concrete. One of the most important factors of making technological selections is weighing the tradeoffs between the short, medium and long term outcomes and costs - the business case must take into account not just the immediate cost but the long term costs including elements that are not immediately quantifiable in monetary terms without expertise.

I unfortunately have some experience on the subject of RPA from a few sources and my motivation for this is to give a word of warning for those contemplating kickstarting capability based on sales hype that still lives in various conferences and domains.

Overview

RPA (short for robotic process automation) is essentially UI based automation - treating the user interface of an application as a programmatic interface (you could liken it to programmatic APIs). But while APIs are designed to be used for integration (and naturally breaking changes are avoided - or with good practices at the very least made deliberately and communicated well) there is no corresponding expectation with the UI since totally different expectations are made of the end user (a human with an ability to dynamically adjust to minor or even major changes in conditions - and to think outside the box).

This paints the background for what are both the strengths and the weaknesses for RPA as a technology. And what, via a multitude of different interactions and systemic side effects, causes the various long term effects which are not immediately apparent from the plain mechanisms for people inexperienced in the domains where such things are routinely handled with tens of years of experience (i.e. IT development).

Sales angle

RPA sales is very often targeted at business and leadership personnel at companies and IT's involvement is downplayed and de-emphasized - it's easy to see why. This factor plays into the commentary on many of the points below.

In RPA sales the over-simplified quick wins angle is over-emphasized while disregarding the large varieties of sometimes non-obvious mid- to long term cost factors (combined with the target audience's lack of insight on these non-obvious long term consequences).

The good

These are the things which show up on hype slides and have some truth to them:

• Specifications don't necessarily require deep understanding of system internals or technical matters at any level because the implementation can happen purely against UI layer which is also how an operator would use it as well
• In cases where systems are old, rarely updated and it's highly impractical to expose modern APIs, the UI-based integration route might be the only option
• In case of very simple processes the implementation time could potentially be quick
• Assuming that the underlying systems play nice with the RPA tech which is not a given (see more below)

The bad

These are the downsides that most sales personnel pushing RPA also recognize and thus are mostly baked into realistic plans.

• RPA is not built to be maintainable. With the focus most often on immediate happy path results the maintainability of the processes is at best highly questionable and at worst a total nightmare
• Since vendors don't even consider communicating their internal implementation details regarding what happens behind the visible UI layer and minor changes in it, RPA processes essentially will randomly explode after seemingly minor or trivial updates that a human wouldn't even recognize as having contained a change
• This means that if the underlying systems get frequent updates then the RPA process maintenance can be expected to cause a major cost factor and headache. This is mitigated by the underlying systems not changing
• The ugly part of this is that in an ecosystem with a large variety of RPA processes this starts inhibiting actual system development and updates
• The outcome being reduction in agility in the longer term
• Failure situations in the underlying systems are a multitude and considering these in the RPA processes is possible only to a degree - and usually these end up being handled in highly inoptimal fashion. See below for more on the topic
• This leads to extensive testing requirements
• ...and bugs frequently still popping up in production. Further exacerbated by potential underlying system changes

The ugly

This is the part with the not so intuitive downsides that often only raises its head when it's too late to do anything about it.

• While very often it's listed as a benefit that no coding skill is required to be able to implement RPA processes, the outcome of this is the following:
• This means that very often people without a mindset for evaluating different execution paths (and no traditional development experience) are selected as RPA developers
• This leads to what everyone in IT knows to expect from junior developers who are not provided good guidance and mentoring i.e. frequent bugs, deficient exception handling, re-implementing basic patterns in non-ideal fashions, creating overtly complex structures that are difficult to maintain
• All of this leads to significant costs in the medium and long term while they may provide the benefit of getting simple happy path processes seemingly running in the short term
• This is further exacerbated by the next point:
• Compatibility of RPA with underlying systems is highly variable. While some systems play well with RPA automation, some systems contain frequent bugs which for users appear as minor annoyances (button at different place, minor naming glitch with text, etc.) but to RPA process cause it to totally stop working since the path selectors (fundamental building block of RPA automation) are often very specific
• Obviously your mileage may vary depending on underlying systems, OS, direct or VPN-based connection etc.
• Overall the RPA tech is very rickety and breaks very easily. This forces the processes to be restricted to ones without significant mistake potential and require additional overhead in the form of checks and balances and building the processes to be repeatable in cases of a significantly high percentage of execution attempts ending up in failure because "something weird just went wrong"
• We're learned through hard experience that there are certain things that must be done to make software (and RPA processes are just that despite the hype) reliable and high quality.
• Some of which are the following:
• Version control with clear diff and pull review for another person to sanity-check & approve changes
• Automated testing at unit and process level
• Automated deployment pipelines
• Monitoring, logging, automated observability to help with potential error investigation
• For all of these cases RPA usually provides basically no tooling out of the box and even makes it actively difficult to create these quality-increasing aspects
• RPA processes themselves cannot really be tested without actually running them. Sure you can manually change the process to only run a certain part of it (and this is 80% of development activity in trying to find the bug relating to some selector or another) but automated testing and verification is not really that's supported
• This leads to longer iteration times at low development level
• And increases the likelyhood that only happy path is covered, yet exacerbating the multitude of issues from all the other mentioned sources

Outcome

In an environment with modern development practices and for the most part functional APIs introducing RPA can only lead to net negative outcomes considering the variety of factors above.

For environments with very old, stable and rarely changing legacy systems representing a significant percentage of the integration landscape RPA can provide a benefit but you still need to consider ways to mitigate the variety of downsides listed.

Which one are you?

Related material

ThoughtWorks' TechRadar (an opinionated expert review at technological trends) has this to say about RPA - giving it the recommendation "Hold" - i.e. do not adopt:
https://www.thoughtworks.com/radar/techniques/rpa

Speed Without Discipline - a Recipe for Disaster Highlights - or Why Automated Testing of the Right Kind is Essential

Another great talk by Venkat Subramaniam on writing maintainable code, how to be truly agile in the long term (and not just short) centralizing on declarative style and automated testing.

https://www.youtube.com/watch?v=uQ75fI1tqoM

Highlights:

• We talk about agility but the faster we go, the more important it is to have discipline or everything turns into an unmaintainable mess
• There's a paradigm shift going on from imperative to declarative programming
• Declarative = intent separated from implementation. The what from the how
• Functional programming is just a sub-set of declarative programming
• Automated testing or rather automated "verification if the code does what it should" is essential
• Proper testing is getting insight into how the application behaves and how it could be developed. This has pretty much nothing to do with verifying compliance to intent - that should all be automated
• "Testing: Act of gaining insight"
• "Mostly UI level verification is "pathway to hell" automation"
• The wrong way to build testing - the cone of test automation from hell

• Mostly automated GUI / end-to-end level testing
• Few service and unit level tests which
• The right way - solid foundation of test coverage at the right layers - the pyramid of automated testing:

• Large number of unit tests - the level at which test writing and execution are both very lightweight and maintainable and can cover a very high percentage of various edge-cases and failure scenarios (in addition to the various happy paths)
• The higher you go the more time it takes to create and execute a test case and the more resources are required for automated test case maintenance
• "I want the test to be at the right level for the right reasons"
• Why is it hard to write testable code? Programming often is a series of mini-experiments that lead us to discoveries.
• If you're doing something for the very first time and it's not clear yet what you're building, don't write test first
• How to overcome this?
• Spike to learn - test stuff, prototype and experiment on a separate little project until what you should be testing is no longer a total unknown anymore
• "Automated verification is the software equivalent of exercising" - requires discipline and mostly none of us do it as much as we know we should be doing because it's hard

Thinking Architecturally

Another absolutely excellent highlighted talk, this time by Nathaniel Schutta covering a large number of different essential topics relating to architectural thinking and decision making. One core skill for any architect or developer is seeing tradeoffs everywhere and looking at and evaluating options from multiple different perspectives. It's very easy to fall prey to technology infatuation (this is awesome!) or the opposite (this sucks!) without critical analysis of fitness for the actual use case.

https://www.youtube.com/watch?v=d5bNZX8tpiI

The talk had plenty of excellent insights, many of which were new to me. And it's always positive to have on-point dry humor to liven up the talk and make the points more memorable.

Some of the best highlights:

• Attention is a resource. It does not scale
• By definition you have to be selective, by definition you cannot read it all
• Architecture is all about tradeoffs but very often it just goes round and round with "on the other hand this on the other hand that". To paraphrase Harry Truman "Please someone give me a one-handed technologist".
• But then again: "Any decent answer to an interesting question begins, "It depends..."" - Kent Beck
• Every tech has drawbacks, acknowledge the negatives
• Interview question (about anything, tools, platforms, etc.) - what would you remove?
• "You haven't mastered a tool until you understand when it should not be used" - Kelsey Hightower
• Create architectural guardrails to align many developers towards the same goal without taking away the option to solve the problem at hand in a more optimal fashion
• We cannot predict the future. "I don't make many predictions... but I do study history.." James Watters
• Adapting to change in architecture - Martin Fowler's tips:
• 1. Build for now
• 2. Choose tech based on ability to evolve
• 3. Evolve one use case at a time 
• It's important to promote a culture of experimentation and constant learning, trying out new tech and prototyping

So many of the points of the talk's more human elements resonated with me. I can often notice myself trying to do it all, trying to learn it all while obviously being impossible I just end up spending time in a very in-optimal fashion. Time is a critical element also regarding decision making - we're forced to make decisions based on incomplete information all the time and building the mental tooling to excel at that is another of the most critical skills (pretty much in everything at life).

Tech radar

Considering the limited time and attention we all have, the best concrete tip for me was mentioning ThoughtWorks' Tech Radar which I wasn't familiar with before (many of you probably are).

In short Tech Radar is an annual / semi-annual publication by ThoughtWorks (known for many best as the place where Martin Fowler is the chief scientist) which provides an opinionated view (in the best of ways) into specific techniques, platforms, tools etc. that developers and organizations should definitely adopt, asses trial or ignore. These suggestions come with brief analysis and rationale as to why that is the case which provide contextual information also on how to mitigate the downsides.

You can find it here: https://www.thoughtworks.com/radar

Do not walk away from Complexity, Run

Second excellent talk by Venkat Subramaniam on the topic of complexity which I'm sure you're heard tons about already. On the other hand it's one of those topics that bears refreshing every now and then since all of us get comfortable and just let it slide from time to time (and let the accidental complexity settle in). I entirely agree that one of the core skills of any developer or architect is the ability to distinguish between essential and accidental complexity

Nowadays it already seems an age-old adage that functional approach very often is a good tool for fighting complexity and it gets highlighted here as well. This is also discussed more in yet another talk by him (also extremely good) on refactoring your language portfolio.

Do not walk away from Complexity, Run:
https://www.youtube.com/watch?v=nZcLHkORdHE

Refactor your language knowledge portfolio - we are what we can code in:
https://www.youtube.com/watch?v=zajUPJI19ZQ

Some highlights combined from both of the talks:

• "Mutability needs company, it often hangs out with bugs"
• Using a library is like dating, using a framework is like marriage
• "Don't build what you can download (library / dependency). Don't download what you really do not need"
• "We looked at this problem and said we can solve it using a pool of threads. Now we have a pool of problems to deal with."
• A key design skill we need to develop is the ability to discern accidental complexity from inherent / essential complexity
• Imperative code is packed with accidental complexity (see examples in the talk)
• What causes complexity:
• Too many moving parts
• Invisible changes
• Uncontrolled mutability
• Lack of cohesion
• Excessive dependencies
• Technology infatuation
• Low level concurrency
• Imperative style
• Entwinement 
• "Good code is the opposite of legacy code: it captures and communicates essence, while omitting ceremony (irrelevant detail). Capturing and communicating essence is hard; most of the code I have ever read fails to do this at even a basic level.
• "The illiterate of the 21st century will not be those who cannot read and write, but those who cannot learn, unlearn, and relearn" - Alvin Toffler
• "There are risks and costs to a program of action, but they are far less than the long-range risks and costs of comfortable inaction" - JFK

The essence of many or even most of the tools regarding handling complexity come down to enabling developer to only look at one piece of a software or enterprise architecture and understand it without having to hold a multitude of other blocks in the puzzle in their mind. In other words the ability to reason about the code or architecture. Understanding this driving force enables you to come up with various tools and techniques to strive towards less complex future yourself without only relying on externally provided tidbits.

Qualities of a Highly Effective Architect (or developer)

Found this excellent keynote talk yesterday by Venkat Subramaniam titled "Qualities of a Highly Effective Architect" that covers the subject extremely well. The same criteria also apply to any developer - not just those shooting for architect position of some sort.

Very rarely does this sort of inspirational type of speech manage to have so insightful content as well.

https://www.youtube.com/watch?v=QeKheNfO3Yg 

Some highlights and quotes I found particularly refreshing and insightful (or just fun):

• Be a mentor, not a tormentor
• Architecting is evaluating tradeoffs
• Nurturing knowledge and good attitude is your professional responsibility
• Worst thing for company is for someone to say "I know this so this is how we do it"
• Must be able to talk about both sides of the issue, not just go "this is awesome" or "this sucks"
• Guide, don't dictate
• "Two sets of people scare me: those who can't follow instructions and those who can only follow instructions"
• "Funny how some ideas go from brilliant to stupid within minutes of prototyping"
• Prototypes beat any argument
• Diversify your knowledge portfolio - learn entirely new things all the time, something that you're uncomfortable learning - that's when it's something actually new. Comfort is the enemy
• Learning happens in deltas - the more entirely different things you've learned previously the faster you can learn something new / evaluate new concepts and ideas
• Lead by example - the worst kind of architect is the power point architect. Architect must be able to code. A good architect practices what they preach. Avoid being a powerpoint architect
• Every technical decision should have an expiration label
• Evolve the architecture - confidence in architecture should always be communicated as a probability - and the confidence should grow with the knowledge of the application
• New tech / migration: first poc, then the hardest problem then the following large mass comes easy 

First post, what's this about?

I have recently been learning a lot about Kubernetes, cloud native development, API gateways, service mesh, hybrid cloud and various other related topics - both for work and out of personal interest. While I do have some previous cloud development experience, I have mostly worked in the more traditional requisitioned VMs / services-as-pets space so there is a lot to be learned to reach an intuitive all-around understanding of everything related to cloud-first development.

The objectives of this blog are manifold: to help myself clarify my thoughts, to have something to link to when discussing these topics, to communicate my ideas and journey to others and perhaps help you learn something as well, get feedback on the gaps and mistakes I've made to grow and sometimes just simply share and summarize some interesting book, article or talk that I encounter around these topics.

I expect that posting here will be very sporadic, ranging between no posts in months to a flurry of activity when I hit a learning phase.

If you learn something from what I've found or from my mistakes, then great! I welcome constructive critique, ideas and thoughts which are the foundation of constant improvement.