Why Data Visualization

Why I chose to pursue a journey in Data Visualization? This post is an explainer to the above question.

What do I do for living?

Data Engineering is my day job. SQL and Python have fed me and kept me going for last decade. For the uninitiated, the work involves fetching data from several sources and loading into desired target. The source and target could be a database, API endpoint, or file.

The type of work is termed as backend work. Or in other words your existence is hardly noticed by anyone. The workflow revolves around CLI (the same black screen that’s shown in hacking movies), some browser tabs and some tools calles SQL client. Date Engineering involves dealing with boat load of data (aka Big Data).

I was not airdropped into the field as Data Engineer. Until 5 years ago, I was called as BI developer or ETL developer, both are interchangeably used. Irrespective of the term, the work involves fetching data from source to target.

You don’t see the difference between the Data Engineer & BI developer? Try harder. Still couldn’t find any? That’s okay. The difference between the roles are harder to detect. The primary difference between a DE & ETL developer is the amount of data they deal with. DE deals with huge volume of data through distributed computing and cloud.

Who will be using our work?

Whatever a ETL Developer does, the end result is consumed by Report developers. Similarly, the outcome of DE’s work is utilised by Data Scientists, Data Visualisation Specialist or Report developers.

Data is a Commodity

In the era of IoT and cloud computing, there is no limit on how much data one can generate or store. Through the DE activities, the data is being moved across systems, cleaned and validated against preset rules and handed over to next team to get some insight out of it.

A commodity is a basic good used in commerce that is interchangeable with other goods of the same type. Commodities are most often used as inputs in the production of other goods or services. Source: investopedia

The same article goes on to say, a product, on the other hand, is the finished good sold to consumers.

Historically, the countries/companies that concentrates in commodity trading reaches a tipping point and beyond which it fails/goes kaput.

Value-adds yield more returns

A company exporting bauxite would receive lesser returns than the company that (apply value addition) convert bauxite into Aluminium. Similarly, comopany that converts Aluminium into airplane would get highest among the three companies.

Likewise, staying as a Data Engineer may not help me in a long run. To get into next level, I have to add some value to exisiting data.

What are the options we have?

For a Data professional, there is a huge specturm of roles are available to pursue. Some of them are,

• Data Analyst
• Data Visulaization Specialist
• Deep learning Engineer
• ML Engineer
• and so on.

This space evolving so fast and keeping tab on new developments and understaning them itself requires some effort. Yet, all of them thrives on data generated by DE.

Among the list, Data Analyst is the one that everyone understand. This requires crunching data and get some insight out of it. DL & ML Engineering is having a quie a lengthy learning curve. Also, it requires good math exposure. Forecast/prediction that results in this work may be tougher to explain to the business people, due to the threshold to understand the inner workings of the underlying algorithms.

Why Data Visualization?

On the other hand, Data Visulaization is an art of building visuls from the available data and conveying insights through visual medium. This is more like story telling with the data. Everyone likes to hear stories. The threshold to understand the visuals is lower, as the abstract concepts are rare or non existent.

To make things clear, Data Visulaization is not something that I’m going to pursue leaving the Data Engineering. It is more like widening the scope. More like a company trying to cultivate a new product line, with the core business intact. The returns from the new product may take time, but its a worthy bet.

See also

• Index Movement - Visualization