Big Data in 2018

15th January 2018 by Paul Newbury | Data

Big Data in 2018

Predictive modelling, neural networks, statistical analysis, R and Python.

These are not new concepts by any stretch. However, what is (fairly) new is the willingness, and the growing ability, within businesses of all types and sizes to adopt these technologies and techniques in an attempt to drive their business performance forward.

I have no doubt that 2018 will see the continuation of these trends, with businesses much more willing to move in this direction and to implement new ideas off the back of this work. This will include investing in training for their staff (and in buying services from agencies and freelancers) to support the implementation.

The real change here can be attributed to the industry having brought these more advanced techniques into the mainstream. There are now reams of training materials available to would-be learners in this area, with DataCamp and similar online learning platforms bringing learning opportunities to the masses.

The trend is also being seen in more formal training offerings; Yard currently delivers a Level 4 qualification in Data Analytics. Other universities and colleges are following suit accordingly, in some cases offering the same qualification and in others, tailoring their own courses to include some of these facets in an increasingly practical, applicable approach.

Additionally, there appears to have finally been a wide-ranging acceptance of specific languages and technologies for this purpose. While there is plenty that can be achieved in numerous other programming languages, the use of R and Python for these tasks is becoming widely agreed upon as the right approach for business. What’s more, these technologies are easily available to anyone wanting to apply their skills, with online courses providing integrated IDE offerings and Python, R, Rstudio and the majority of libraries available as free software downloads to everybody.

The use of these libraries has had a similar effect on the ability to progress in these areas. Libraries like Shiny in R have made it easier to produce fully productionised systems based on the data wrangling happening in the background. The Keras library in Python makes it very easy to start playing with advanced neural networks with little to no prior knowledge of the underlying techniques necessary (though, let’s be honest, it would be more than useful!).

All told, technical barriers to entry in these areas are being progressively removed to ensure that these advanced tools are available to all.

Yard on Data in 2018

So, what does this mean for business? A key advantage will be the increased availability of experts in this field working to implement these technologies and techniques. Similarly, awareness of their use is much higher within businesses and so support within the industry to continue investing in this area is good.

On the other hand, there are also a number of challenges to be faced:

  • Staff being trained and who upskill in this area are suddenly much more attractive to other businesses, agencies, etc. As such, investing in an individual increases the likelihood that they will eventually leave for pastures new. Nonetheless, we’d always encourage investing in staff and besides, they will want to prove their worth while they are with you. 
  • There is often a disconnect between modelling and action. Predicting individual behaviours is a challenging but achievable task. Identifying the effect of individual website changes/marketing campaigns/product offers becomes a simpler task. However, what does this translate to? What action can we take to ensure the continued success and incremental improvement of results in our businesses?

These (positive) challenges mean that work remains to be done and money to be invested. However, the very fact that we are no longer satisfied with reporting numbers, preferring instead to understand and exploit the data available to us as businesses bodes very well for 2018 and the many years to come.

Leave a Reply

Your email address will not be published. Required fields are marked *