We interfere with artificial intelligence (AI) and machine learning (ML) as part of our daily life. From movie recommendations on Netflix, advertisement targeting on YouTube, biometric facial recognition to much more. AI and ML accumulate a lot of ‘buzz’ and creates excitement but how do you choose the right project? The one that will succeed and generate increased revenue for your company.

In 2017, firms around the world spent over $21.8 billion on mergers and acquisitions related to AI. We are at the beginning of a revolution, as McKinsey predicts that digital innovations will provide an estimated economic impact of about $370 billion per year worldwide by 2025. However, despite the glamour excitement around AI, we must also not be blinded and pick cautiously your next AI project. Despite the positive association, the industry cultivates around this technology. According to Harvard Business Review (HBR), more businesses buy the promise of Big Data and AI initiatives, in fact, they found a growth of 65% from last year. However, Forbes recently reported that many executives worldwide have not seen value from AI investments. So with technology promises, some billion-dollar projects are failing to deliver on their promises, and we don’t want yours to be one of them.

We brought you Hamid, an AI and Data expert to give you 4 tips to choose the right AI project and avoid the pitfalls of another risk that will leave you with nothing but empty promises and money loss.

At integrityCo, our team has over 60 years of collective experience with machine learning, big data, data science, and software development — so when it comes to AI projects, we’ve seen everything. Our mission is to help you to make the most of the data you already have. And as technology grows and we hear about big data everywhere, companies rush to invest their money for growth, we thought it would be helpful to share 4 common pitfalls and their solutions when choosing your next AI project. After you’ll read this article you will be able to choose your next AI project wisely and ensure success.

As social human beings, our success relies on our collaboration with our environment and that’s why our first tip is about teamwork!

1. Team misalignment

All projects, especially ones that move towards human innovation, depend on team collaboration. Misalignment among stakeholders is like fighting an uphill battle.

According to an HBR survey, 93% of respondents claimed that team disagreements and issues in the process of the project were the number one cause of the failure of their projects. With a high turnover economy, the challenges with culture change have been dramatically underestimated. HBR found that 40.3% identify a lack of group organization, and 24% cite cultural resistance as the primary factors stifling business adoption which disrupts the workflow. Consequently, negatively impacting the return of investment for AI projects.

Solution: Communication is key

With the difficulties of working remotely, communication is still the key to success. work can flow smoothly once all the working brains agree on the project strategy. Diverse perspectives are valuable for the team to grow and evolve. Therefore, allowing everyone to voice their opinion is critical to the project’s success. We recommend beginning your project with a workshop. By facilitating an engaging brainstorming session, people not only feel comfortable sharing their thoughts, but it also gets stakeholder buy-in, creates alignment, and gets everyone on the same page which saves time in the long run.

2. Quantity over quality

Data is critical to making informed decisions on AI projects. Often in the interest of saving time, many companies approach data gathering with the mindset that more is better. Unfortunately, this results in teams feeling overwhelmed and unsure of how to parse through hundreds (sometimes thousands) of datasets to determine what is useful. In conducting data analysis for clients, we have come across terabytes that provided little insight into their problem and megabytes that were so rich with information we could build a sophisticated ML project. The key is to ask the right questions when trying to implement an AI or an ML project. That’s exactly what integrityCo focused on. Finding the root of gold mine data.

Solution: Take small steps, and when in doubt, outsource

Little by little, little becomes a lot. To determine relevant data, start with a small and easy exploratory analysis. Use what is readily available to determine if you can pull out any signals. An experienced data scientist should carefully take step by step to identify patterns. If nothing comes out from one path, try a different mindful strategy. Instead of trying to do everything at once, try to do one thing in order to get everything. Be ready to detour, start over and keep being alert to any signal that may emerge along the way.

Developing an AI product is resource-intensive, and for many organizations, it’s challenging to allocate the in-house time and talent to work on every facet. Hiring a consultant with relevant domain and data science experience can be a great investment to ensure you collect quality data PLUS get an external point of view. One that your internal team might have not been aware of or qualified for.

3. Irrelevant KPI + small impact

We’ve seen clients get swept up in fun and interesting projects only to find out the business value is centred around vanity metrics. For example, clients measure an increase in visitors when their focus should be increasing ARPR [Average revenue per referral]. So, before you start pulling your big guns in any project, identify your goals, opportunities and measure of success. Even if a project is tied to a relevant KPI, the effort required to make an impact may not worth the stakes. Passion projects are important, but before taking one on, first, determine relevant KPIs and what would you consider as a goal attained.

Solution: Know your business

To determine relevant KPIs, you need to know your business — this may sound trivial, but many companies can’t articulate the problem they are solving. You want to be able to explain it clearly, fast and so that anyone can understand, guide yourself through this thought process:

• What are we trying to solve?
• For what purpose?
• And what impact does it have on our business? Community?

As you are going through this exercise, think of ROI, not COI (coolness of investment!).

A tip for calculating ROI: ROI can be determined by looking at how much you stand to make or save if the project succeeds. The example below demonstrates how you might go about forecasting ROI for a potential affiliate marketplace project.

Simplified Business Model of an Affiliate Marketplace

Simplified Math Equation of an Affiliate Marketplace

4. Unrealistic goal

Consumer-facing businesses want to make a chatbot that reduces staffing costs. E-commerce shops would like to anticipate what their visitors are going to buy and target accordingly. Pharmaceutical companies would love to make a drug for cancer.

We get it, everyone wants to be the next leader, we live in a very competitive based market but, if your goal is not realistic, it will simply be impossible to attain and your project will fail. So, check-in with reality and acknowledge your work scope. After years of executing data science projects, we’ve noticed a trend. The chance of success appears to be related to the complexity of the problem. The more complex, the less likely it is to succeed.

The more complex, the higher chance of failure!

Solution: Simple is better

Partnering with a consultant or an agency that understands how to make the complex simple is a great way to increase success. If your company’s BHAG (Big Hairy Audacious Goal) is to create a chatbot that reduces staffing costs, break it down into smaller steps.

Thriving in uncharted waters

AI and ML are being positioned as the next great step forward for humanity — but we have a ways to go before society realizes the full benefit of these powerful technologies. Technology is dynamic and constantly shifting. Every day we develop, learn, reveal a new idea. Companies looking to dive into AI and ML projects are navigating uncharted waters and for this reason, you need the sharpest, experienced but also open-minded minds. If your organization doesn’t have the in-house expertise, time, resources, or methodical R&D process, we encourage you to hire outside consultants early on to get going in the right direction. All projects come with a risk of failure, but with the right team, knowledge, KPI and strategies… you will see yourself on the path towards success.

The current coronavirus pandemic situation has created a sudden change in our everyday lives that was unforeseen just a few months ago. As individuals adjust to the “new normal,” businesses should also take a moment to reassess their situation.

Suddenly, the market is upside down, your team is working remotely, and a good portion of your team probably doesn’t have as much on their plate. Although the market is showing signs of coming back, it might take many months to a few years. This uncertainty presents an opportunity to concentrate on the “fundamental” work that’s always in the “to do” bracket. Tasks that were a lower priority with everyone focused on the daily business operations is now perfect for getting done. It’s time to position your company to leverage the current situation so that your company is super competitive in the “new normal.” Well, let me try to explain:

Utilization = morale

We know that excellent staff morale is key to productivity and utilization. I would argue that those who are most engaged are those that understand the reasons for what they are doing. If they are aligned with the company’s vision and understand their roles and possible impact, they will not need your constant motivation. Nothing creates a downward spiral more than underutilization of staff already suffering from the “Mushroom Syndrome.” Now is the perfect time to create alignment and focus. Typically, teams are too busy when things are going well

I’m guilty of this; you’re guilty of this. Typically, when things are healthy, and the company is doing well, everyone is laser-focused on their commitments and targets. We are guilty of working ‘in the business’ rather than ‘on the business.’

The market has changed, and the world is a different place–get ready for it

Some of the effects of COVID-19 are with us for a long time, and some are likely permanent. Now is the time to get ready for dynamic changes that are new to everyone. Markets after the 2009 crash were different. Similarly, the markets after September 11 also changed. The changes after the two events were different too! What do you need to be ready? Data can tell you a lot about this. It would help if you readjusted some of those fundamental assumptions which worked previously. Do you know which ones they are?

New technology initiatives take time

You have got your teams focused on the next 12 months of work; now you need to start thinking of the future. Once you have set the company’s direction, you need to gather intelligence to determine the criteria for success and failure. To quickly identify where the market is headed and stay on the leading edge. The best way of creating a picture is by analyzing many data. That’s where technology can help identify new trends and emerging patterns, but the intelligence lies in understanding data. Implementing technologies like AI and ML takes time. More than that, gathering data takes even longer, and it requires a lot of trials and error testing. Experiment with it while your business is not at full capacity. Now is the best time for when the market comes back, you won’t have the time.

So what?

OK, you’re convinced that it is a good time to do some fundamental work. Where to start? Start from where the impact is. Try to avoid shiny objects and only do what matters. I’m going to write about this topic next week so please stay tuned! Before that let’s look at some other bullet points.

Measure twice and cut once

You’ve heard the saying “Measure twice and cut once,” well, this also applies to businesses. By that, I mean companies should spend time planning their mid to long-term course before committing the workforce to a particular path. Moreover, that same workforce should be part of the planning process, so they have a stake in the outcome.

During this downtime, it’s your job, as the boss, to create the company vision for the 10–5–1 year horizons. You are responsible for the big picture. Where do you see the company (in terms of revenue, size, product development, market capture, and any other target you have been dreaming about) in 12 months from now, then 5 years, and finally 10 years from now. Identify the targets as accurately as you can and see if your staff think it’s possible. If they like what they see, they’ll provide most of the solutions to achieving the ‘want.’

Try filling in the details of your targets in the following table style:

Does everyone working for you know what the company does? Why it exists? You may think those questions are trivial, but to get alignment across the board, everyone must understand what service(s) the company provides, and why it exists. Now is your chance to consider how the current Covid-19 climate can be an opportunity to beat the competition.

In the following diagram, I created a mind map to identify the business of a company. It needs to be simple enough for anyone to understand.

What YOUR landscape looks like?

Start by identifying your current place in the market and then define how new opportunities are becoming apparent. Let your staff tell you what they think.

Along those lines, I created the following drawing for a company to highlight its position in the market. It shows their competitors, suppliers, partners, customers, and other channels. Again, a mind map is the easiest way for everyone to follow:

More efficient and last longer

Use your best and brightest employees to help set the path forward. Once they fully understand what the company does, why it exists, and where it needs to go, they’re motivated to succeed.

Task your leaders with creating milestones to achieve the goals for the next 12 months, let them demonstrate their preparedness to meet those targets. When that occurs, your job is to shepherd them, provide the resources, and trust them.

Finally, it’s very important to find us on LinkedIn and let us know what you think! Always happy to connect.

Our goal is to learn about Naïve Bayes and apply it to a real-world problem, spam detection. Why Naïve Bayes? Consider building an email spam-filter using machine learning. You would like to classify messages according to whether they are unsolicited commercial (spam), or non-spam (ham). Classifying spam vs. ham is one example of a broader set of problems called text classification, a set of problems that Naïve Bayes can solve with relatively high performance. The Naive Bayes classifier is a powerful classification algorithm that works based on modeling conditional probabilities. It is particularly useful for classification problems with many-many features, and as we find out shortly, negligible conditional dependencies. The name “Naïve” suggests that we are making a wrong assumption here; in fact, we are! But as it turns out, our Naïve assumption works pretty well in practice.

import pandas as pd
file_encoding = 'utf8'
input_fd = open('./spam.csv', encoding=file_encoding, errors = 'backslashreplace') #the file has some non utf8 parts
data = pd.read_csv(input_fd)
data = data.iloc[:,0:2]    #removing the null columns
data.columns = ['label','content']
data.head()

from sklearn.feature_extraction import text 
word_bagger = text.CountVectorizer()

word_bagger = text.CountVectorizer(stop_words="english")

words = word_bagger.get_feature_names()
len(all_words)

words[2000:2010]

X = word_bagger.transform(data['content'])

test = word_bagger.transform(['I am happy happy'])

print(test)

from sklearn.naive_bayes import MultinomialNB
spam_nb = MultinomialNB()
spam_nb.fit(X, y)

spam_nb.score(X,y)

ham = "Let me know if it works for you? I will keep you posted!"
spam = "Buy real cheap eyeglasses here, eyeglasses.club!"
test_bagged = word_bagger.transform([ham, spam])
spam_nb.predict(bagged)

What do you think of sales? How do you feel about a salesperson? If your thoughts are not exactly positive, you’re not alone. , the author of To Sell is Human (I recommend you to read it!), asked his readers “when you think of sales or selling, what is the first word that comes to mind?” He took the top 25 most frequently mentioned words and put them in a word cloud. Words like “pushy,” “yuck,” “ugh,” and “sleazy” were viral. Craig Wortmann , a Professor of Entrepreneurship at Kellogg School of Management, asks his sales course students to think of the first words that come to mind, when he says this word, salesperson. “These are the words people say. Aggressive, pushy, persistent, slick, relentless, hustles, sleazy, helpful, and manipulative.” In fact, according to a new survey from HubSpot Research , only a mere 3% of people consider salespeople to be trustworthy.

Why do I care?

Well, as a founder and entrepreneur, a big part of my job is selling. Selling our product, our team, our vision, etc. But coming from a technical background, sales was not and is not natural to me, yet! Having a negative mindset about salespeople, I used to ask myself, am I doing something terrible here by selling? I was self-conscious about it…

After starting my second company, I started to take sales courses with a mindset that, to survive, we need to sell, so let’s just learn how to do it and get over it. I started course by Craig, and it changed my life ! Surprisingly, I discovered I was looking at it the wrong way. As Craig or my sales coaches put it, sales is about helping someone find the solution to a real and vital problem they have. It’s being an advisor. Sales is being a doctor and curing people’s pains.

Why you should care?

Well, regardless of how you feel about it, we are either selling or being sold to all the time. That is a fundamental consequence of being a social animal. Remember that occasion that you started talking to that cute girl? You were selling. Job interview? You are selling. Asking your friend for coffee? You are selling. Presenting your results to your team? You are selling. Sometimes, we have done it so often that we don’t recognize it anymore; but we are always selling and buying.

Can you be a great salesperson?

OK, being able to sell is essential. The question is if you can be good at it? I think you can as long as you care about who you are selling to. lists 10 psychological traits of what makes a great salesperson. Let’s look at the top 5:

1. Ability to Build Empathy

The more mature I get, the more I find how invaluable it is to be empathic. Being a salesperson who knows how to connect, build relationships, listen, and build rapport is hugely crucial.

2. Ability to Listen and be Naturally Inquisitive

It goes hand in hand with empathy indeed. You cannot know how you can add value to someone without listening to them genuinely and of interest. Put your agenda aside and listen!

3. Ability to Put Yourself in Someone’s Shoes

Putting yourself in someone’s shoes is the foundation of empathy, which enables you to form rapport with anyone.

4. Intelligence and Ability to Infer

Opinions and needs are not always stated explicitly. You need to dig deep and dig intelligently. Good salespeople can make inferences and logical deductions, reading between the lines to understand the true meanings and desires of their prospects and be able to elicit pain points strategically.

5. High Emotional Intelligence

Emotional intelligence is what defines an exceptional salesperson vs. a mediocre one. Being able to understand, predict, and act based on what your knowledge of the other side’s feelings is pivotal for success.

Where to start?

OK, you decided that you want to be an exceptional salesperson. Where do you start? There are all of those fancy sales training programs like , Challenger Sales and etc. I’m going to take some of these in the following months and will keep you posted about my opinion about them. Before that, I think we need to work on the top 5 skills and traits listed above. Some of us are more natural in some than the other. However, we can definitely improve by practicing. I used to be not very good at listening. After becoming aware of it, I challenged myself to develop in that area, and today I can say that I am great at it! So here are some resources to become better in each:

1. How to Build Empathy?

Read this and watch this.

2. How to Listen and be Naturally Inquisitive?

Read this.

3. How to Put Yourself in Someone’s Shoes?

Read this and this.

4. How to Ask intelligence questions to Infer?

Read this.

5. How to Have High Emotional Intelligence?

Read this.

What next?

I’m going to continue writing on this subject, so stay tuned! Also, please let me know if you found this helpful; it makes me happy to see that I am helping you. My email is [email protected].

We asked one of our clients, a well respected and rapidly growing company in the Vancouver Sales Vertical what impact our AI engine had on their Sales Operations. They told us that within four months we had reduced their sales team cost by 20% while at the same time adding more than $6 million to their projected annual revenue.

We were quite taken aback. Initially we expected a more modest impact as our Machine Learning Solution was relatively new and our algorithms were not fully trained. In fact we had not even begun training our model to predict churn and lead scoring.

This feedback led us to reach out to other firms like Maximizer, FreshBooks, etc and when we asked them about their toughest challenges in their sales and sales operations, we consistently heard the same messages:

• Duplicates in their CRM systems were causing them frustration and the lack of verified contact information was wasting valuable prospecting time.
• There was unhappiness with having to scale up a sales team when it was possible to enable already trained staff to work more effectively.
• Many leaders mentioned to us that although there was a surplus of AI solutions for marketers, they were disappointed that so few effective machine based learning solutions existed for sales and sales operations.
• Others commented that they were frustrated at managing vast troves of data knowing that they were missing out on key insights and having to spend way too much time managing their ever growing data piles.

Over the next few weeks we will explore deeper into these topics. Sign up for our newsletter, as we share our findings around:

• What parole applications can teach us about timing in sales.
• How to calculate the real costs of CRM duplicates
• How AI can enrich your data beyond basic demographic data
• Which smart-bots work best with Machine Learning to optimize your lead generation
• Why a 5% increase in your lead scoring is the best thing to accomplish this week
• How correctly matching a lead to the right salesperson makes for happier customers and a more engaged sales force.

Stay tuned!

We recently helped a client to clean up their Salesforce CRM. First, we made our proprietary solution to find all of the duplicate records that would pass the client’s criteria and then had to use Salesforce native merge functionality to remove the duplicates. By doing the deduplication process, we helped them save their Sales Reps time by an estimated 10% to 20%.

After a few hours of searching online, we figured that there is not much documentation on how to interact with the Salesforce API we had to work with, their SOAP API and particularly no one had talked about using their merge functionality. This article is written to make it easier for other developers that are planning to do what we did and save them time, a lot of it. Let’s say Salesforce API and SOAP APIs are not the most fascinating APIs to work with!

Our Programming Language:

You can send a SOAP request through any programming language that you like. Here, we are using python as it is becoming more and more popular especially among data scientists and other data related experts. Extending it to your favourite language should be quite straightforward though.

A Bit of Background:

Let’s look at a bit of background on SOAP protocol that can be found on Wikipedia:

“ SOAP (abbreviation for Simple Object Access Protocol) is a messaging protocol specification for exchanging structured information in the implementation of web services in computer networks. Its purpose is to provide extensibility, neutrality and independence. It uses XML Information Set for its message format, and relies on application layer protocols, most often Hypertext Transfer Protocol (HTTP) or Simple Mail Transfer Protocol(SMTP), for message negotiation and transmission.”

So essentially, we need to send an xml message to Salesforce API that is based on a given standard. You need to look at the Web Service Description Language of Salesforce (WSDL) to (hopefully J) find out how to format your xml message. The good news that we’ve done this for you for merging.

Let’s Get Started

First, we need to make a nice header to send with our API call. Here is how the header should look like:

headers = { 'Content-Type': 'text/xml', 'Accept': 'application/soap+xml,application/dime, multipart/related, text/*', 'Authorization': 'Bearer ' + self.session_id, 'SOAPAction': 'merge', 'Sforce-Auto-Assign': 'false', 'charset':'UTF-8'}

It should be quite self-explanatory; if you are wondering about ‘Sforce-Auto-Assign’: ‘false’ part, it’s helping us to avoid any weird default assignment rule that your Salesforce might have.

How about the body?

Now it’s time to actually make a nice xml message. We are doing a bulk merge request and hence we have a list of records that we want to merge. We have stored that list into a list called data and each row of data is a list containing the following in order:

• sf_type: type of the records to be merged (Account, Contact, Lead)
• master_id: Salesforce Id of the master
• victim_id: Salesforce Id of the victim
• valuesToKeep: Please see below

For example, the first row of data can be:

['Lead', '00Q2G00001atpfx', '00Q2G01001atqfl', {'Industry': 'Law', 'Phone': '+1 111 111 1111'}]

One very important note. Salesforce does not keep the info from the victim even if the fields for the master are empty! You need to tell it explicitly to overwrite those fields. As a result, we made a dictionary ( valuesToKeep) of field-names (keys of valuesToKeep) and victim values (values of valuesToKeep) we want to overwrite into the master record. For example, in the above, we are asking Salesforce to use ‘ Law’ as the final record’s ‘ Industry’ and ‘+1 111 111 1111’ as ‘ Phone’.

Finally, let’s put all of these together and form xml body:

body = '''<?xml version="1.0" encoding="utf-8" ?> <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:enterprise.soap.sforce.com" xmlns:urn1="urn:sobject.enterprise.soap.sforce.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><soapenv:Header> <urn:SessionHeader><urn:sessionId>{sessionId}</urn:sessionId></urn:SessionHeader> </soapenv:Header><soapenv:Body> <ns1:merge xmlns:ns1='urn:partner.soap.sforce.com'>'''.format(sessionId = self.session_id) for rec in data: sf_type, master_id,victim_id,valuesToKeep = rec body = body + '''<ns1:merge><ns1:masterRecord> <ens:type xmlns:ens='urn:sobject.partner.soap.sforce.com'>{sf_type}</ens:type> <ens:Id xmlns:ens='urn:sobject.partner.soap.sforce.com'>{master_id} </ens:Id>'''.format(sf_type = sf_type, master_id = master_id) for rec in data:
    sf_type, master_id,victim_id,valuesToKeep = rec
    body = body + '''<ns1:merge><ns1:masterRecord>    
    <ens:type xmlns:ens=
    'urn:sobject.partner.soap.sforce.com'{sf_type}</ens:type>    
    <ens:Id xmlns:ens='urn:sobject.partner.soap.sforce.com'
    {master_id}
    </ens:Id>'''.format(sf_type = sf_type, master_id = master_id)    for item in valuesToKeep.keys():
        try:
            value = valuesToKeep[item]
            body = body + '''<ens:{item}   
            xmlns:ens='urn:sobject.partner.soap.sforce.com'>
            {value}</ens:{item}>'''.format(item=item, value=value)        except:
            passbody = body + '''</ns1:masterRecord><ns1:recordToMergeIds>{victim_id}</ns1:recordToMergeIds> </ns1:merge>'''.format(victim_id = victim_id) body = body + '''</ns1:merge></soapenv:Body></soapenv:Envelope>'''

Let’s send our request!

We use the library requests to the API call. You are free to use another method that suits you.

import requests 
from simple_salesforce import Salesforcesession = requests.Session() 
sf = Salesforce(username = username, password = password, security_token = token, session = session, domain = domain)sf.session.request(method = 'POST', url = 'https://' + sf.sf_instance +'/services/Soap/u/' + sf.sf_version, data = body, headers = headers)

You need to provide for username, password, and security_token. domain is ‘test’ for Sandbox and None for the other cases.

Note: Salesforce API version 38 is problematic. You want to avoid it.

If everything goes well, Salesforce returns you a xml that contains no error message. Otherwise, it will try to help you understand where it went wrong, and you can correct any potential mistakes.

I wish you good luck and enjoy working with Salesforce API! Feel free to contact me if you had any questions.

If you’re here, you definitely have heard about data science and you might wonder what data scientists really do? There is no good answer to this question! Data science is a relatively new field that was coined in 2008 by D.J. Patil, and Jeff Hammerbacher, then the respective leads of data and analytics efforts at LinkedIn and Facebook. As the field Is getting more mature the definition of data science changing drastically. The plot below used to be a popular way explaining what data science is. A data scientist used to be someone who knows a bit of math, , machine learning, statistics, programming and some domain expertise.

Nowadays, new jobs such as machine learning scientist, data engineer, applied researcher and … have popped up affecting what we expect from a data scientist. Today, “more than anything, what data scientists do is make discoveries while swimming in data. It’s their preferred method of navigating the world around them. At ease in the digital realm, they are able to bring structure to large quantities of formless data and make analysis possible. They identify rich data sources, join them with other, potentially incomplete data sources, and clean the resulting set. In a competitive landscape where challenges keep changing and data never stop flowing, data scientists help decision makers shift from ad hoc analysis to an ongoing conversation with data” [1].

What makes data scientist different than data analysts?

Data analyst and data scientist skills do overlap but there is a significant difference between the two. Both the job roles require some basic math know-how, understanding of algorithms, good communication skills and knowledge of software engineering.

Data analysts are masters in using tools to slice and dice the data. With some level of scientific curiosity data analysts can tell a story from data. A data scientist on the other hand possess all the skills of a data analysts with strong foundation in modelling, analytics, math, statistics and computer science. A data scientist is expected to generate their own questions while a data analyst finds answers to a given set of questions from data. A data scientist is someone who can predict the future based on past patterns whereas a data analyst is someone who merely curates meaningful insights from data.

But we would say the dominant trait among data scientists is an intense curiosity-a desire to go beneath the surface of a problem, find the questions at its heart, and distill them into a very clear set of hypotheses that can be tested. This often entails the associative thinking that characterizes the most creative scientists in any field. For example, we know of a data scientist studying a fraud problem who realized that it was analogous to a type of DNA sequencing problem. By bringing together those disparate worlds, he and his team were able to craft a solution that dramatically reduced fraud losses [1].

Why data science is getting a lot of attention?

Data is considered as modern gold [2]. The sudden appearance of data science on the business scene reflects the fact that companies are now wrestling with enormous amount of data that comes in varieties and volumes never encountered before; they need to extract information from that data to take actions and benefit from. Many organizations store multiple petabytes of data and the data most critical to their business resides in forms other than rows and columns of numbers. Those organizations are in desperate need of data scientist to go through their data and make sense of it to stay ahead of their competition.

What it means for you?

If “sexy” means having rare qualities that are much in demand, data scientists are already there. They are difficult and expensive to hire and, given the very competitive market for their services, difficult to retain. There simply aren’t a lot of people with their combination of scientific background and computational and analytical skills. It’s a great time to enter the market and become a data scientist so hurry up; if you have the foundations, gain the skills you need to be a good data scientist and start a sexy career!

1. “Thomas H. Davenport and D.J. Patil, Harvard Business Review, Data Scientist: The Sexiest Job of the 21st Century”
2. “The Economist , The World’s Most Valuable Resource is no Longer Oil, but Data”

“We are drowning in information and starving for knowledge.” — John Naisbitt.

The Era of Big Data

We have entered the era of big data. For example, there are about 60 trillion web pages; 300 hours of video are uploaded to YouTube every minute, amounting to 10 years of content every day; the genomes of 10000s of people, each of which has a length of 3.8 × 109 base pairs, have been sequenced by various labs; Walmart handles more than 1M transactions per hour and has databases containing more than 2.5 petabytes (2.5 × 1015) of information [1]; We are currently creating around five exabytes a day roughly equivalent to 500 million songs — the amount of data available today is giving machines the possibility to become super-intelligent.

What is ML?

This deluge of data calls for automated methods of data analysis, which is what machine learning provides. In particular, machine learning (ML) is defined as a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data) [2]. A more technical definition of ML is “a system’s ability to correctly interpret external data, to learn from such data, and to use those learnings to achieve specific goals and tasks through flexible adaptation” [3].

AI vs ML?

It’s worth noticing that often ML is used with artificial intelligence interchangeably. Although artificial intelligence is technically broader than ML and they are different, for the purpose of this note, we treat them the same.

Why Now?

Now, you might ask, how old is this stuff and why is it getting a lot of attention now? Artificial intelligence was founded as an academic discipline in 1956. It because most of ML methods work well when you have a large amount of data (to be precise it really depends on the problem you are trying to solve) and as noted we have entered into an era that there is no shortage of data, yay! Another interesting thing recently happed; our computers got way stronger and faster than before. Analyzing and extracting information from an enormous amount of data is not trivial. It needs a lot of computational power and the introduction of graphics processing units (GPUs), which were developed to speed up the graphics in our everyday computers and especially in the gaming world, to ML world changed the game.

What are Some of the Applications of ML?

What are some of the applications of AI/ML? Good question. Although you might not notice it, ML already is a big part of our lives. Amazon makes its recommendations based on a set of ML algorithms called “Recommender systems” specifically a hybrid of “Collaborative filtering” and “Content-based filtering”. Google uses ML to detect spam emails. Facebook uses ML to show you the content that you would find interesting. Tesla is using AI/ML to make self-driving cars. Intelligence services are using ML to detect criminals in crowds and …

ML and AI are already here. The question is how to align your business to benefit from it rather than getting killed by it.

1. “Cukier K., The Economist, Data, data everywhere: A special report on managing information, 2010, February 25, Retrieved from http://www.economist. com/node/15557443”
2. “Kevin P. Murphy , Machine Learning: A Probabilistic Perspective Hardcover — Aug 24 2012”
3. “Kaplan, A. and M. Haenlein (2019) Siri, Siri in my Hand, who’s the Fairest in the Land? On the Interpretations, Illustrations and Implications of Artificial Intelligence, Business Horizons, 62(1)”.

The first time I felt the gap was during my PhD years when one of my close friends was looking for a job. He is a smart guy with great talent and a PhD in computational physics from a top university, UBC. He connected with many great companies whom really needed his expertise and had 5 successful interviews with each of them. However, they never hired him, as they thought he did not have enough job experience! They preferred to hire a candidate with a BSc with a couple of years of work experience than a recently graduated PhD. He started a tech company instead and after a year and half, had a successful exit.

There is a massive disconnect between leadership teams of companies who mostly come from non-technical backgrounds and the technical world, let alone academia. The gap is quite dangerous for the existence of these companies. We live in an era where technology plays a crucial role in companies’ success or ultimate failure. The leaders of a company need to have accurate understandings of the transition of the technologies used in their sectors and other sectors. They need to understand how they can benefit from the new technologies available and how they should plan for the future of their business, given that what used to be science fiction would be a reality soon.

In this post, the emphasis is on artificial intelligence (AI), machine learning, and data science and their potential contribution to a company’s success or failure. Until now the main beneficiary of AI has been the technology sector. However, more non-tech companies in a broad range of industries are starting to worry that AI could scorch or even incinerate them. In 2017, firms worldwide spent around $21.8 billion on mergers and acquisitions related to AI. Around 85% of companies think AI will offer a competitive advantage, but only 5% are “extensively” employing it today. This provides plenty of opportunities for the first-movers and substantial threat to others. McKinsey predicts that digital innovations including AI will provide an estimated economic impact of about $370 billion per year worldwide in 2025, so how would you benefit from this exponential opportunity?

Based on my personal experience, most business leaders have limited knowledge about the potential of AI for their business. Worse, their companies do not have experts who can provide consultation. Even the companies who have teams of data scientists and/or machine learning scientists cannot benefit from the knowledge of their team as the two sides live in very different worlds. Technical teams are mostly concerned about the details of projects and do not focus enough on the main concerns of the business team and the OKRs.

Now what can you do to make AI work for your business? McKinsey has a good report on this. I definitely suggest reading it. Here are some of my additional recommendations:

• Get an AI Advisor: Having advisors who are experts in a given field is a great way of making the right decisions. It is a common practice for startups to have advisors as they don’t have the resources to hire enough senior talent, but I believe it is a fantastic way for any company to stay competitive and informed about the new opportunities. There are many advisors who have strong technical backgrounds, but as a result of their business experience have learned the language of business teams. They can communicate effectively with the C-level team and even play the role of a bridge between the technical teams and the business teams.
• Attend/Organize Conferences/Meetups: No matter how experienced you are at your business, everyone can learn and it is really easy to learn new stuff in a more technical conference. Conferences gather a wide range of expertise with different ongoing projects. The speakers spend hours preparing for talks and the topics of these talks can give you a good sense of the technology trends in your industry and the other sectors. Meetups gather a more diverse set of attendees with different goals for attending the events. In addition to the usual networking opportunities and getting your brand out there, more technical meetups attract “nerdier” attendees, resulting in interesting conversations.
• Find an Expert Partner: You do not need to do it on your own. With so much interest in AI, it can still be really hard to attract good talent in that domain. As the demand for more hands-on consulting projects is increasing, more consulting firms are being born, which are focused on developing customized AI solutions. Think of them as an advisor that helps you to build the solutions in contrast to what McKinsey says. As not every company needs an IT department, not every company needs a team of AI/machine learning scientists.

Whatever you do, don’t sit still and start your AI transformation today.