6 Reasons to Use Azure Databricks Today
6 Reasons to Use Azure Databricks Today. Global Leader in delivering success with Business Applications based on the Microsoft Cloud.
Read the BlogDo you know if your Azure environment is set up in the most cost-effective configuration?
If you answered ‘No’, it is probably time to find out. It’s true, the Azure cloud can reduce the costs and overhead required to manage your assets, but it also creates the potential for a great deal of waste and inefficiency. Having virtually unlimited compute resources at your disposal means your organization must always be wary of excess costs that don’t provide business value.
Unfortunately, most organizations are unprepared to profit from the Azure savings opportunity and are likely to overspend, according to Gartner, Inc., in a recent article on managing and optimizing public cloud costs. This is because cloud services demand a fundamental change in how you think about resource costs and consumption, and they require a tight collaboration among the disciplines of governance, architecture, operations, finance, and development.
Microsoft well-architected framework
Developing a cost management strategy that projects and plans for the consumption of cloud services are a crucial starting point. Architects are sometimes oblivious to cost when they design cloud solutions. As a result, they may make bad design choices, or changes in application features over time may have rendered a design choice unexpectedly expensive.
The Microsoft Azure Well-Architected Framework has five aligned and connected pillars, and one is cost optimization. All your efforts toward optimizing costs should be grounded in the framework’s principles to help produce and maintain a cost-effective cloud architecture and achieve business objectives. The point is to architect with costs in mind — making the wrong design choices from the outset can be very expensive and sometimes difficult to pinpoint on your monthly billing in the future.
Costs of cloud computing
It’s easy to overspend when you don’t know what you’re spending and what it’s being used for. At its simplest, the cost of cloud computing can be put in three buckets:
- Compute— the amount of CPU resources and memory based on per-second usage of the machine types, persistent disks, and other resources that you select for your virtual machines. You pay according to the number and types of instances used and for the duration they are used.
- Networking— the volume of data transferred in and out of the cloud service. There may be special charges for virtualized network services such as static IPs, load balancers and VPN gateways. If you aren’t using these services, the main cost to consider is VNET peering, which allows you to link your virtual networks.
- Storage— the volume of data stored. For elastic storage services, you’ll pay for actual usage. For managed storage services, like disks attached to compute instances, you’ll pay for the entire storage volume regardless of the amount you use.
Consumption-based pricing
Azure offers different payment plans so you can choose what suits you best. The most common one is the pay-as-you-go option, or consumption-based billing, where service and product charges are determined by the amount of resources or services you use. These services or resources are made up of transactions, CPU time, or the application’s run time.
This model can be cost-efficient because you aren’t paying for the resources to run your application when it’s not being used. Because nothing is reserved, the cost of running the resources grows along with demand, so you must be hyper-vigilant when monitoring.
Other pricing options
Reserved instances
Reserved Instances (RIs) let you commit to a fixed number and size of VMs and sizes for a period of one or three years. You can pay upfront for the committed usage, or monthly for better cash flow management.
In exchange for that commitment, you’re billed at a fixed rate for compute or VM usage (the longer you reserve, the better pricing you get). Reserved instances can potentially provide significant cost savings over the consumption model. However, you need a clear picture of your historical and forecasted spend to make it a good fit. Reserved instances might be best suited when you have a fixed budget and an application with stable ongoing usage, like an application with the same number of virtual machines in use.
Spot pricing
Spot pricing allows you to bid for available capacity on the Azure marketplace and receive instances with huge discounts. However, the catch is that processing can be interrupted on short notice with no prior notice, so few workloads can use these types of virtual machines. Spot instances are ideal for distributed, fault-tolerant applications, or stateless applications that aren’t time-sensitive. A prime example of this is Azure Databricks.
Hybrid benefits
If you have existing on-premises Windows Server and SQL Server licenses with active Software Assurance or subscriptions, you can reuse those licenses in Azure at Linux pricing. It’s a take-your-license-with-you option, and Microsoft says you can save as much as 85 percent over standard pay-as-you-go rates. But just like with the other pricing options, many factors will determine your actual cost savings. You need the correct licenses to qualify, and savings will depend on usage, location, and your existing infrastructure.
Common cost issues and fixes
At Hitachi Solutions, we’ve helped dozens of clients evaluate, assess, and optimize their environments. We see a host of problems that cost clients unnecessary expenses during those engagements, and some crop up repeatedly. At the top of our list:
Paying for unused resources
If you have resource workloads used only periodically but are running continuously, you’re wasting money. These resources can be suspended when they’re not in use, saving on compute costs during this period.
Demo or sandbox resources that are no longer in use can be the biggest culprits. Another example is a development environment left running 24/7, when development generally happens only during business hours.
The impact of eliminating unused infrastructure is relatively high and can often have an immediate and tangible impact on your business’s cost savings.
Improperly sized resources
Along with unused resources, overprovisioned service allocations are among the top contributors to public cloud spending waste, according to Gartner in a recent article on managing cloud costs. This is where ‘right-sizing’ comes in.
Right-sizing resources focus on maximizing the use of existing compute resources at the lowest possible cost — it’s the process of matching resource size with the requirements for resource demand. Unfortunately, many new Azure developers often deploy much bigger resources than needed, leading to unnecessary expenses that are not cost-effective.
Using virtual machines (VMs) as an example, reviewing metrics like the disk, memory, and CPU can reveal the best way to right-size your resources. Additionally, if a VM is running more than 10 percent idle, reducing the size of the VM will immediately reduce your cost. Virtual machine costs are applied within an instance family; each next size larger will double your cost. Conversely, reducing a VM by a single instance size will reduce your cost by half.
Poor tagging
Poorly tagged resources can cost you money. Resource tags are a way to apply additional metadata on your cloud resources to add information that isn’t included in the resource name. Once properly implemented, it is easy to govern and manage resources based on descriptive metadata tags and track related expenses more precisely.
For example, you could use resource tags to manage resources assigned to a specific project – even when project resources are spread across multiple resource groups or subscriptions. Additionally, by adding a tag to identify the project, you can quickly delete all related resources when the project is finished and no longer needed. If you have a project using 100 databases, a tag that tracks those database costs would certainly highlight cost opportunities that would otherwise remain hidden.
Resource tags can also be helpful when you’re running different workloads in separate environments (test, development, quality assurance and production). You could also tag resources by business area to better associate and align budgets by business need. Tags may then be exposed in billing reports for transparency and cost allocation.
The Hitachi Assessment
Hitachi Solutions’ cost optimization assessment is a unique advisory service that can identify and reduce cloud waste, all while supporting the Microsoft well-architected framework for cost optimization. We take a step back to evaluate your existing setup through a discovery process, so we can make clear and actionable recommendations to increase performance and affordability. We’ll tackle the issues that make the most significant impact first and give you a roadmap and tools to evolve.
For every dollar you spend, this solution returns double that back: you can save demonstrably on costs.
State-of-the-art report suite
Azure itself can identify waste, visualize your costs, and make reserved instance recommendations, but at Hitachi Solutions, we extend the capabilities that Azure provides with our custom-made report suite. We take your cost data one step further, so you get a more detailed analysis for a more fine-tuned optimization.
For more detailed information about the specific reports, you can check out our solution sheet. But here’s an illustration of the forecasting report that allows you to predict future cost by subscription, resource group, category (storage, compute, database, VMs etc.), resource tag, or any other dimension you can imagine.
Self-serve access
Access to the cost and usage-related metrics is part of educating your users and goes a long way toward increasing accountability. It’s also an essential piece of the optimization puzzle.
You can use Power BI to connect to the Hitachi report suite, providing self-service access to your organization’s cost usage data. With Power BI, you can then create custom reports and measures to better understand and analyze your Azure spend. It’s another set of capabilities for further slicing and dicing with interactive dashboards for better visualization.
Getting started
Architectures are never static: resource demands shift over time, and cloud services continually evolve to introduce new features and cost savings.
This article is an excellent primer for explaining Azure costs and common cloud costing issues so you can answer:
With the power of Azure and the resources of Hitachi Solutions, optimal cost performance isn’t unattainable. You can get there. It starts with a tenured partner who can define metrics for success, make them easy to run and accessible to all users, and can adapt the right cloud strategies as your organization grows and standards change. Start maximizing the business value of Azure now.
Reach out to us today!