Ready for 2020: The Rise of the Data Architect
Kirk Ryan
It is sometimes hard to grasp just how much digital data as a planet we generate each day. From small contactless coffee purchases at a Starbucks to hours of CCTV footage; the digital footprint is growing rapidly and almost exponentially. In 2012, a study by IBM estimated that 2.5 exabytes - that's 2.5 billion gigabytes (GB) - of data was generated every day. Jump forward just five years, and another study found that the data created on mobile devices alone had shot up to eight exabytes per day and is still growing.
For enterprises of all sizes, digital data has always been useful, but it is increasingly the most important fuel to power line of business activities. And for some, an entire business model is based on how quickly data can be collected, processed and turned into action.
Historically, data storage was hived off into a silo within the enterprise IT department. The big four wings of applications, servers, networking and storage were distinctly different skill sets, suppliers and driving goals. The storage team talked of SAN and NAS with a focus on adding continually more capacity and performance to meet the demands of new applications – while bemoaning the lack of faster network connectivity.
This paradigm still endures in a few places, but the last decade has seen enterprises become more agile in an effort to capitalise on the benefits offered by the internet and to stave off rivals as markets have opened through globalisation.
The arguments about the benefits of the cloud are now redundant
If you look in the most progressive of enterprises, the storage team are no longer an isolated outpost concerned with the minutiae of data management but are increasingly having to transition skill sets to integrate with the wider goals of the enterprise. The most pressing change is the move to the cloud. The arguments about the benefits of the cloud are now redundant. For many workloads, it is the most effective option and especially for developers building the next generation of apps that focus on areas such as machine learning; the sheer flexibility on offer is hard to ignore.For IT staff that have had various storage-related job titles, the next big transition is to the role of data architect. However, where the shift from direct attached to NAS and SAN was technology focused, the role of a data architect encompasses not just learning new storage technologies but more critically how to deliver value through data to the rest of the organisation.
This means gaining a fundamental understanding of the benefits and limitations of the cloud model. This also means being part of key projects such as information governance in relation to regulatory constraints such as GDPR. The data architect now gains a seat at the table for many more enterprise activities because they are elevated from a traditional infrastructure role to a more intrinsically valuable position that enables a wider range of business critical tasks. This also includes more grandiose projects that could well shape the future direction of the business for the next decade.
NetApp Cloud Data Services offers the familiarity of on-premise
For many, the starting point of the transition will be either replicating or moving large portions of data storage architectures from on-premise data centres into cloud equivalents that are fit for purpose to match the business need. In most cases, the skills and technologies that are well understood for an on-premise storage architecture can move seamlessly into the cloud with little effort. For admins familiar with NetApp platforms, the current generation of NetApp Cloud Data Services offers the familiarity of on-premise while being built from the ground up to take advantage of cloud-centric capabilities such as integration with containers and the block, file and more recent object-based storage technologies that are beloved by the likes of AWS and Azure.The skills that are needed for a transition to a data architect role should at least encompass gaining certifications of the preferred cloud platform of the organisation, whether that’s Amazon, Microsoft, Google or whoever. As somebody that has been down this journey, it is less daunting than you would imagine. Best of all, the cloud platforms have very similar core architectures which makes becoming conversant across all the big public clouds a relatively easy progression.
Having skilled data architects is becoming one of the most critical requirements
However, the move towards a data architect role should not just be viewed as just a personal desire. Enterprises have a vested interest in elevating the data team to a more front and centre role to support an increasingly agile IT environment. There is a clear skills shortage alongside a fear that can be articulated in a quote often attributed to the entrepreneur, Richard Branson; “Train people well enough so they can leave, treat them well enough so they don't want to!”With data considered by many enterprises as the most valuable commodity they possess, more so than gold and oil, the next decade will require both technology and people that are ready for the transition. In a world where cloud is not just an option, but the first port of call for many organisations, having skilled data architects is becoming one of the most critical requirements. The organisations that are ready to start the journey sooner rather than later are in the best place to reap the benefits.
Kirk Ryan
Kirk Ryan is a senior cloud solutions architect who regularly provides consultation for successful adoption of cloud first strategy and cloud architectures. Kirk is experienced in AWS and Azure design methodologies, serverless technologies, IoT systems, analytics and machine learning, web and mobile development (Node.js, React, React-Native, GraphQL) as well as over 14 years of data management and solutions integration experience. He is the leader of the UK Cloud meetup, and a frequent speaker at events and executive briefings
