Web Scraping: Getting new clients
What does web scraping mean to you?
Well, I didn't actively seek it out, I can assure you of that! I've been writing about attracting new clients to my business, and one of the methods I've tried is to search for businesses in a specific location and category. While this approach has been somewhat successful, it's not scalable. I need to be smarter about it.
To start, I asked ChatGPT for a list of the categories that Google uses for businesses. I received a list of 4000 categories. However, this is too many to work with, so I decided to focus on "translator" and "Switzerland" for now.
When I searched for "translator in Switzerland" on Google Maps, I got an endless scroll of translation companies. This is not helpful for building a list of potential clients.
So, I asked ChatGPT for advice on how to make the process easier and learned about web scraping.
Wikipedia describes web scraping as:
Scraping a web page involves fetching it and extracting from it. Fetching is the downloading of a page (which a browser does when a user views a page).
So, down the ‘let’s learn a new technology’ rabbit hole I went.
Why am I doing this?
Yes, this might be a good time to talk about that. As I mentioned at the top, the current method of approaching potential clients is not scalable, although a lot of fun. I needed a more efficient way of approaching many businesses with the expectation that a small number might actually even notice me. So, in came cold emailing.
Again Wikipedia:
A cold email is an unsolicited e-mail that is sent to a receiver without prior contact. It could also be defined as the email equivalent of cold calling.
I want three email chains in my business.
Cold email: mass emails sent to a list of prospects with the expectation that a small number will react to the message.
Warm email: manageable list of prospects where a more personalized message is included including one or two insights I have to offer.
Direct email: a small number of prospects also includes those who react to either the ‘cold’ or ‘warm’ emails. This would include specific insights I offer them and other services I can offer.
You're all familiar with this: spam filling up your inboxes. To ensure success with cold emailing, I need a large number of potential clients' email addresses. Sure, I could purchase lists from online brokers, but why spend the money and give up on a challenge? So, how did this challenge proceed?
Tom Sawyer Travels Newsletter
How did my web scraping go?
Two days into the rabbit hole I had learned a number of new things and a few on the border of legality. But no fears, everything I did here and mention in this post is all legal.
I mentioned that I have a list of 4000 Google categories. These categories are part of the Google Business Profile. Any business can have such a profile. If you have seen ‘stars’ next to a business while searching on Google then you have seen a Google Profile. The ‘category’ is just one of the fields in the profile.
I had already manually searched for ‘translators in Switzerland’ so now I applied my new skills and ‘scraped’ Google Maps for the same. Within 5 minutes I was presented with a list of 1090 translators in Switzerland.
Ok, now this is a number I can start working with.
There is just one problem with this list. Well, actually there are a few problems with the data but one glaring gap; there is no email address on Google Maps.
I knew this was the case as I manage my own business on Google Profiles. I also knew that the company’s website can be part of the profile and through the URL I could get to the website and see if there was an email address. Problem solved!
After filtering the 1090 items for those only having valid websites I reduced the list to around 700 businesses. How do I get the email through the websites for 700 businesses? Web scraping comes to the rescue!
Again, using the data I had filtered I ran the ‘website scraper’ and waited. This took a long time. On some websites, the email might be found on the homepage; on others, it could be buried on page 8. The scraper pulls up a maximum of 10 pages from a website and if it finds one it captures that email address and moves on to the next business. So, potentially between 3000 and 5000 webpages needed to be crawled and this takes time.
After about 3 hours I had 596 email addresses. I manually found another 12 from those where it was not successful. I now have a list of close to 608 translators in Switzerland.
Now what?
Yeah, good question and the answer will be in next month’s newsletter. So don’t miss it!
But to give you a peek, here is the idea.
I want to have 3 offers that I put into these emails. This is pretty standard marketing psychology 101 stuff (I did interview a marketing behavioral psychologist once).
A link to my free eBook “Save Time & Make Money: 20 Online Growth Hacks to Expand Your Business”
A discounted offer for an Online Health Check of their business.
The last one would be to engage me for a one-on-one session over 4 one-hour sessions where I teach them how to improve their online activities to grow their business.
Web scraping can be a valuable tool for businesses looking to attract new clients. By using web scraping techniques, businesses can efficiently gather email addresses and other information from potential clients, allowing them to implement effective email marketing campaigns.
As I continue to refine my own marketing strategy, I look forward to seeing the results of my efforts and sharing my experiences with you.
