Today, for many companies, Cloud computing has become an alternative for on-premises servers. Apart from cost savings, the main drivers of cloud adoption are data recovery, archival, and ease of management.
Why businesses are leaning towards the cloud?
Cloud is no longer risky or untried or untested. It has matured and is now highly reliable. According to IDC, the US is the undisrupted leader in terms of cloud computing spending. The projected spending of the US for cloud computing in 2019 was $124.6 billion.
Following are the reasons for the increased adoption of cloud today:
Agility: Using on-premises servers limits an organization’s performance to the resources that it has installed. System performance can be highly impacted in the case of unforeseen growth in resource demand. Whereas, with cloud, companies don’t have to plan hardware upgrades to meet growing demands. They can avail and manage resources instantly.
Resiliency: With replicated storage, and the ability to harness multiple network providers, cloud can greatly enhance disaster recovery, business continuity, and availability. Leveraging cloud, organizations can avoid the risks associated with a single private data center. Since high availability is inherent in cloud, users can meet arbitrary resiliency requirements at a lower price. The benefits of cloud were exhibited in real-time when the COVID-19 pandemic compelled almost all organizations to function remotely.
Scalability: Cloud offers access to unlimited resources virtually. With multiple data centers available in every region, organizations can resize their systems as per their workloads and new demands.
Efficiency: Using cloud, resources can be deployed in times of demand and turned off again when unnecessary. Metering and automating resource usage not only helps organizations control spend and wastage but also maximize efficiency.
Security: Cloud service providers like AWS, Azure, Snowflake, Google Cloud Platform are required to meet the industry-set highest security standards. Also, authentication, authorization, logging, and auditing is in-built in all cloud platforms. Since the organization is in complete control of its cloud environment, cloud is not any riskier than on-premises architecture.
Cost-efficiency: Owning and maintaining on-premise data centers require a significant amount of capital as the infrastructure encompasses space, construction, electricity, physical security, hardware procurement, operating systems, human resources, and much more. Shifting to cloud services can enable organizations to dramatically save money by cutting costs on maintenance, management, and training.
Best practices for migrating from On-Prem to Cloud
Moving from legacy to the cloud infrastructure is an end-to-end transformation around data, infrastructure, and security. Following are the best practices for migrating to the cloud:
Step 1: Migrate existing data
The first step is to select the right cloud data warehouse and create an initial copy of the organization’s existing data in it. Organizations will have to experiment with different cloud infrastructures to choose the appropriate one for them. Migrating smaller data sets across different data warehouses will help them in comparing and selecting the right one. Also, migrating hundreds of terabytes of data over the internet to the cloud is not an easy task. It is a good thing that Google and Amazon offer various means like transporting hard drives, trucks, USPS, etc, to reduce the complexity in migration and make the process simpler.
Apart from copying the raw data, the format and schema of the data exported from the warehouse should also be verified. Then, import the data’s schema into the cloud data warehouse before loading the data. Most importantly, mark the point in time of the exported snapshot as it is essential for the next process –establishing an ongoing replication mechanism.
Step 2: Set up ongoing replication
Once the first snapshot of the on-prem data warehouse is taken, copy it into the cloud data warehouse to establish an ongoing synchronization process. An ongoing replication mechanism is more complicated compared to a single copy operation, as it entails a series of incremental copy operations. Each operation will capture any changes to the data and its schema, say deleted data, altered column types, etc, and replicate the changes in the cloud data warehouse. More technical challenges are likely to occur in this process. Hence, any synchronization solution should be gauged for reliability and latency. These parameters are highly important to the success of cloud migration.
Step 3: Migrate BI
Analytics infrastructure forms a major component of an organization’s data infrastructure. Appoint a small innovative team to lead the change. Leverage BI tools that will befit your cloud data warehouse. Some of the best BI tools are Tableau, Charito, Redash, Periscope, Looker, etc.
Step 4: Migrate Legacy data application
At this point in the migration process, the organization will have a cloud data warehouse that runs 5X more reports than the on-prem data warehouse. After migrating BI, prioritize the migration of data applications and custom reporting tools. The process is likely to evoke many technical challenges as you identify the ODBC drivers and data models that need to be changed and the queries that need to be re-written, in order to fully benefit from the performance advantages of the cloud data warehouse. Leveraging data synchronization mechanism can help with the implementation of data model changes.
Also, incorporating various data engineering teams will help in building a rollback safety net and rolling out newly migrated applications effectively.
Step 5: Migrate Legacy ETL process
The final step is the migration of the ETL process to the cloud data warehouse. While some instances may require just a change of configuration, others may require a thorough re-write. Thankfully, Cloud ETL provides many comprehensive services that will make the process less tiresome.
Although companies will face teething problems during the migration, navigating through the challenges and adopting cloud is worth it, particularly for the performance and cost-efficiency benefits.
Use Cases
According to Cisco, 94% of workloads will be processed by cloud data centers in 2021. Following are a few companies that benefitted from cloud migration:
- Spotify: A media services provider that mainly focused on its audio streaming platform, Spotify, had difficulty in provisioning and maintaining its massive in-house data centres. The situation worsened to an extent, where the company couldn’t free up its engineers to focus on innovation. This affected the company’s key objective of being the ‘Best music service in the world.’ Ultimately, in 2015, they started planning for migration to Google Cloud Platform (GCP). The on-premise to cloud migration enabled Spotify to increase scalability and free up their developers for innovation.
- Betabrand: A crowd-funded and crowd-sourced retail clothing e-commerce company, Betabrand, struggled with its bare-metal infrastructure. Maintenance and scalability were the key sore points for the company. Even more so, planning for and adding capacity was time-consuming and cost prohibitive. However, the anticipation of 2017’s Black Friday, encouraged the company to migrate to cloud. The migration was indeed a success and enabled them to handle peak loads during Black Friday sales, with absolutely no issues. Furthermore, since the migration, the company hasn’t experienced a single outage.
- Evernote: A cross-platform note-taking and task management application, Evernote, has owned, configured, and maintained its servers and network since inception. However, with the bare metal infrastructure, Evernote started feeling increasingly limited as it was difficult to scale. Also, maintenance was expensive and time-taking. After an intensive period of research, in 2016, Evernote opted for cloud platform services from Google. Since the migration, Evernote’s experience changed and it became robust, stronger, faster, and more stable. The company was able to complete critical infrastructure upgrades that used to take days, in just seconds. Also, they could manage resources effectively to improve core products and accelerate current developments.
Final Thoughts
With the exponential increase in data use and the upsurge of web-enabled devices in today’s business environment, Cloud has become the need of the hour for organizations to operate at the required levels of efficiency, security and scalability.
