On What We Diagram in Data and Tech

And why I sometimes favor the burlesque over the simple

As technologists in data, we are forced to not diagram reality.
Sometimes, that’s a problem.

Simple

Data technology vendors, authors, thought leaders and architects are caught in a dilemma when explaining their ideas: To make technology understandable, their diagrams have to not depict reality - reality understood as how the technology would be applied in an enterprise context. The motivation is honest: We all need simple diagrams to understand complicated technologies. And yet the problem is, that the reality in which these technologies will be used, is not simple. We are left with a paradox - the simple diagram is both a needed didactic illustration and at the same time an unrealizable ideal.

It’s impossible to explain technology without the diagrams, in they can be considered reference architecture. The component in a reference architecture carries out a clearly defined, specific capability.[1] And yet, that reference architecture is something we refer to, in discussions about how reality should be. But never is.

However sometimes, we need to favor diagramming the real reality - not the reality as it should be but the reality as it actually is. And that reality is not simple.

Burlesque

Here is one of the first illustrations in my upcoming book Fundamentals of Metadata Management.[2] It’s crowded. It’s messy. I think of this diagram as burlesque. Burlesque because there is something intriguing about it, it’s a sort of friendly mockery of the perceived rationality of technology, that almost always materializes in weird, unforeseen ways - creating a reality that is both opulent and dysfunctional, and yet works.

The problem is, that if we do not diagram the reality of technology usage, in a context that is wider than simply the smallest possible one in an ideal reference architecture, we are unable to address the reality of metadata management. We are somehow left with only one choice of diagramming: A not very persuasive perspective for decision makers, a sort of theoretical as-is diagram of what is the reality in companies world wide. And yet, for metadata repositories that depict the IT landscape, this type of diagram is how we can push forward and address reality, not as it ought to be, but the reality as it is. Only on that basis can we change it.

Notes

[1] The Open Group: The TOGAF Standard, Version 9.2 (Van Haren Publishing, 2018), see Reference Model

[2] Ole Olesen-Bagneux: Fundamentals of Metadata Management (O’Reilly, forthcoming 2025)