Melbourne Business School builds big data capacity with Databricks

Students speaking

Deloitte Access Economics pinned Australia’s data science population at just over 300,000 in 2018 – but demand was forecast to grow 2.4 per cent a year through to 2021-22, significantly outstripping overall jobs growth. More than three quarters of organisations planned to increase their investment in data science.

The writing was already on the wall five years ago when Melbourne Business School (MBS) started offering its advanced one-year full time degree that focuses on quantitative methods and the role of data in decision making. Students in the Masters of Business Analytics course use state of the art technology and develop the skills that employers crave.

The challenge for MBS was to provide students with a computing environment that allows them to tangle with big data, and perform the sort of analyses that are required for real world problems.

A cloud based digital transformation at MBS – which has seen the institution deploy Microsoft Azure, Microsoft 365 and Dynamics 365 – fortuitously established the scalable computing foundations that have allowed it to also offer business analytics students the opportunity to work on very large data challenges that simply would not be possible with their own laptops or on-premises technology.

Simon Holcombe is academic director for the Master of Business Analytics course; he has a PhD in theoretical physics and a decade-plus experience in the financial services sector in roles that were heavily analytics dependent. That combination of intellectual rigour and industry experience gives him a clear vantage point regarding what students need to achieve in order to meet enterprise needs.

He says students taking the Masters; “Get a rigorous introduction to statistics, computer science, business analytics and business acumen.” The course, he says, is designed; “With the direct purpose of feeding top notch graduates into industry,” who will rapidly be snapped up for senior roles.

Modules in the Masters course run from introductory subjects through baseline statistics and data warehousing analysis. As the student progresses, more advanced subjects, such as machine learning and business analytics are covered.

MBS this year deployed Azure Databricks to provide students with a platform able to handle big data analytics – converting what was one theoretical teaching into practical hands-on learning.

Using Jupyter notebooks students access Azure Databricks which acts as an easy to use interface to Apache Spark environments which are designed specifically for big data and machine learning analytics challenges.

“You might use big data to do sentiment analysis in emails to look for customer churn, or on a huge segmentation analysis,” says Holcombe.

Azure Databricks has underpinned a big data project at MBS which sees students take Yelp’s publicly accessible restaurant data, analyse that and then use the resulting insights to craft a business pitch to set up a new restaurant based on a fine-grained understanding of what diners like, in which geographies.

Students can run analytics to identify trends, clustering patterns, to perform textual analysis of reviews, and find which cuisines are popular in which geographical areas to then identify business opportunities.

While the Yelp data collection is modest (16 GB) compared to some corporate data collections, its analysis would still be too large for most computers. It’s a walk in the park for Azure Databricks.

Holcombe says; “Anyone who wants to successfully lead a team of analysts in the future needs intimate knowledge of tools and techniques of data, statistical analysis, machine learning and some familiarity with the infrastructure that drives those analytics.

“This gives them that opportunity. They will take that in to industry and will become more sophisticated – but they have been exposed to it.

“Further down the line, if they get into senior strategic roles, having this knowledge from the ground up gives them more insight and understanding of what analytics can do,” he says. Critically, the experience will also instil in students a deeper understanding of how business should adopt analytics.

“Everyone is creating machine learning models – but are they appropriate? How do you know they are appropriate? How do you formulate the questions?,” says Holcombe.

“A key aspect we try to instil in students is that nothing is more important than the business problem so that they are utilising the right tools in the right way to get outcomes that are useful to the business.”

While MBS considered analytics solutions from both Microsoft and AWS, the Azure Databricks solution stacked up best in terms of maturity and capability. According to Holcombe; “The solution from Microsoft fit the bill in all ways and wasn’t complicated, it was easy to implement, easy to use for educators.

“And MBS’ natural affiliation with Azure meant that it was a cleaner solution,” with students’ and academics’ Azure Databricks access managed by Azure Active Directory.

While Azure Databricks has been used initially in the Masters of Business Analytics course, Holcombe foresees other opportunities for example in text and web analytics courses, or for predictive analytics in economics. “It is a natural tool of inclusion throughout the curriculum,” he adds.


Our goal at Microsoft is to empower every student to achieve more and to do this we have a range of resources available to support educators in achieving better learning outcomes for all students. Find the right Windows devices for your students, deploy Office 365 Education with ease,  join our Educator Centre for free training and connect with others, or attend one of our MS Edu Meets or IT IQ Edu Meets workshops.

Follow us on Facebook and Twitter for our latest updates.