A Brazen Expose on Big Data in the Government Sector

Picture of Lillian Pierson, P.E.

Lillian Pierson, P.E.

Reading Time: 5 minutes

A few weeks back, I was contacted by Tamsin Rutter of The Guardian’s Public Leaders Network. She asked me if I would like to participate in Guardian’s online debate about big data in the government sector. Given my experience with this, I was more than happy to oblige.

There were about 10 experts present and active in the debate. You can read the re-cap here if you’re interested to know more about what everyone else had to say.  I found the debate to be a bit challenging. Tamsin didn’t actually ask the questions that she had provided me by which to prepare. For this reason, I have used the questions that he originally provided me as a guide post here in my Brazen Expose on Big Data in the Government Sector.

***Before proceeding, please heed the following warning***

In this article I am only discussing my personal experience. I have made some pretty broad and condemning generalizations. Many of them may not be representative of what is happening in your business or organization. Maybe your IT department is really cool and has tons of great data scientists and analysts. Maybe you are lucky enough to work for an organization where everyone is collaborating and helping each other out. If so, consider yourself fortunate! Based on my experience and many conversations I have had with fellow data analysts, data visualization specialists, and data scientists, you are in the minority. The content of this article only expresses my opinion. It is based on my experiences and the experiences of my colleagues who’ve practiced data science in the public sector.

And now, without further ado… let’s begin!

Which existing parts of central and local government are best-placed to use (big) data effectively?

I would say that all divisions could stand to benefit by using (big) data effectively. The problem is often in resistance to change. Managers generally aren’t operating from a quantitative perspective. They are people managers, not data analysts. Getting them to adopt technology for data-driven decision-making is often the largest obstacle. It’s a “if you build it, they probably won’t come” scenario. To drive adoption of the data-driven decision-making approach, you’ll likely need to get that mandated by organizational leaders-at-large. And even if you do, adoption does not mean “effective adoption”. Managers have to want it for (big) data to be effectively used in government.

But I digress… I think that any direct customer service line of business within the government has a lot to gain by adopting (big) data and analytics to improve customer experience. Internal business operations could also be made a lot more resource efficient. But again, enforcing effective adoption of (big) data in government is a very difficult thing to do.

What changes are needed to effectively pull together (big) data in the government sector?

In many governments, as in many private sector organizations, there is still a significant obstacle to overcome the “silo” effect. Not only that, you have classic IT departments out there that act as a true hindrance to any sort of progress with respect to (big) data or its effective utilization. I think the classic IT department is the biggest hindrance and the biggest thing that needs to be changed in order to facilitate the incorporation of (big) data and the business insights it generates.

IT departments classically serve a vital role! They make sure the infrastructure stays secure. They make sure that the infrastructure is adequate to support operational demands. IT performs a number of vital tasks, many of which I am unfamiliar with because I have never worked in proper “IT”, per se. This said, classical IT department might be populated with great IT Engineers and Computer Engineers and whatnot. But they’re generally not populated with great analysts.

IT is not known for its strong skills in “Data Science”, “Data Analytics”, or “Data Visualization”. That’s generally because these roles are outside the range of their expertise. They are not statisticians, analysts, or data scientists. Instead, they are IT engineers. They host and secure business data. But they don’t have the substantive knowledge to make sense of that data.

Making matters worse, classical IT departments are generally VERY threatened by government (internal) businesses having coders and programmers working on and making sense of their data.

IT loses control in these situations… and it’s a decentralization of “computing skills” to entities that are operating outside of their control. They often thwart efforts to do this by any means possible… so (big) data technologies are many times “banned” for no justifiable reason… just because IT department says so.

For this reason, (big) data and its insights cannot be pulled together to benefit most government (internal) businesses. Until this changes, I honestly don’t see a way that (big) data can be used effectively in the government sector (at least in governments that face this same dilemma). Again, the answer to this is in the organizational managers mandating and ENFORCING changes to be adopted, this time in the IT line of government business.

What are some ways of approaching (big) data differently, through events such as hackdays?

Hack days are great! Open data is great!! It’s so inspiring to see what is being done in New York City. This said, it’s hard for me to imagine many government organizations TRULY making their data “open” like NYC has. There are political risks in doing that… and the IT departments would see that as a huge risk and vulnerability. There is a lot of talk about “open government”, but I think that governments are usually only as “open” as they need to be to check that box off the list of PR requirements to qualify as a “cool” government”.

If government employees can’t even freely use and access the data that their line of business is generating, there is no way in heck that public citizens would get unfettered access to it. Governments have SO MUCH to gain from the work of data-do-gooders and digital volunteers who are eager to use “open data” to improve their communities… but governments are slow to adopt, and unless the citizen base knows and deliberately pushes for REAL “open data” and “hack days”… I’m afraid that many of these things will be little more than PR exercises.

What is the potential impact of good leadership and senior champions of (big) data in the public sector?

Good, strong, data-savvy leadership is the answer to the government (big) data woes (almost all of them I would say). Government leaders are the only ones that have the power to mandate and enforce effective adoption of (big) data driven decision-making at the managerial level. Because there is a steep learning curve, and because most managers are not that tech-savvy…. Leaders will have to push hard (both indirectly through inspiration and directly through enforcement). Leaders will have to mandate IT to let internal business analysts, staff, and managers have and use their data freely… No more of this stalemating ad nauseum so that nothing effectively gets done. There is going to be a HUGE HUMP and leaders are going to have to push hard to get their organizations to the other side. Passive leadership will never due.

This said, when the changes are finally propagated through, government is looking at a huge payload in the form of resources and taxpayer’s dollars saved. Considering the deficit spending of most governments, it’d be nice if the push for adoption comes sooner rather than later.

Interested in Reading More?

If you would like to read more on applying innovation and technology to help governments deliver better, more customer-centered products and services, then you can check out Customer Service.Gov: Technology Tools and Customer Service Principles for Innovative and Entrepreneurial Government. If you want some different perspectives on civic big data and how data-do-gooders are working towards making a utopia of the public sector, more information can be found on that in Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia. (Please note, if you decide to purchase one of these books, I receive a small commission based on that sale.)



I’m a fractional CMO that specializes in go-to-market and product-led growth for B2B tech companies.
If you’re looking for marketing strategy and leadership support with a proven track record of driving breakthrough growth for B2B tech startups and consultancies, you’re in the right place. Over the last decade, I’ve supported the growth of 30% of Fortune 10 companies, and more tech startups than you can shake a stick at. I stay very busy, but I’m currently able to accommodate a handful of select new clients. Visit this page to learn more about how I can help you and to book a time for us to speak directly.

Get Featured

We love helping tech brands gain exposure and brand awareness among our active audience of 530,000 data professionals. If you’d like to explore our alternatives for brand partnerships and content collaborations, you can reach out directly on this page and book a time to speak.

Join The Convergence Newsletter

See what 26,000 other data professionals have discovered from the powerful data science, AI, and data strategy advice that’s only available inside this free community newsletter.

Join The Convergence Newsletter for free below.

Our newsletter is exclusively written for operators in the data & AI industry.

Hi, I'm Lillian Pierson, Data-Mania's founder. We welcome you to our little corner of the internet. Data-Mania offers fractional CMO and marketing consulting services to deep tech B2B businesses.

The Convergence community is sponsored by Data-Mania, as a tribute to the data community from which we sprung. You are welcome anytime.

Get more actionable advice by joining The Convergence Newsletter for free below.

See what 26,000 other data professionals have discovered from the powerful data science, AI, and data strategy advice that’s only available inside this free community newsletter.

Join The Convergence Newsletter for free below.
We are 100% committed to you having an AMAZING ✨ experience – that, of course, involves no spam.

Fractional CMO for deep tech B2B businesses. Specializing in go-to-market strategy, SaaS product growth, and consulting revenue growth. American expat serving clients worldwide since 2012.

© Data-Mania, 2012 - 2024+, All Rights Reserved - Terms & Conditions - Privacy Policy | PRODUCTS PROTECTED BY COPYSCAPE

The Convergence is sponsored by Data-Mania, as a tribute to the data community from which we sprung.

Get The Newsletter

See what 26,000 other data professionals have discovered from the powerful data science, AI, and data strategy advice that’s only available inside this free community newsletter.

Join The Convergence Newsletter for free below.
* Zero spam. Unsubscribe anytime.