What I learnt from my first beta test
TL;DR summary at the end
After working for two years on my SEO SAAS, it was time to get real feedback, I entered the beta without knowing what to expect, I threw myself in the water, not knowing what in store.
I put a simple message in two Facebook groups (Lion Zeal, White Hat SEO): “Hi People, I’m looking for beta testers for my SEO product, it’s similar to Ahrefs/Semrush only lighter, I will provide full functional user for the beta testing”.
I wasn’t expecting what happened next, I got about 50 replies for each post over the next 24 hours, so I messaged every person that said he’s interested, the first thing I discovered is Facebook starts to show you captcha and warn you after you send about 20 messages, didn’t expect that, so I had to add the following to my post: “Apparently Facebook limits the number of messages I can send, send me a Facebook message and I’ll add you”.
I did eventually get to send a message to all the people that replied, I did it over time to avoid any penalties from Facebook. After my addition to the post some people sent me a message, some replied to the post.
Another tip is that there’s a message request tab in Facebook messenger and there’s also an option to see filtered result, many messages arrived to the filtered part.
At first I sent people “Hi”, but I then realized that if some hours passed between my message and their reply to my post, my hi could look like spam, so I adjusted the message into two options, those whom I directly messaged soon after their posted: “Hi, It’s about your comment for beta testing from the seo group”, and those whom I messaged after a while: “Hi, It’s about your comment for beta testing from the seo group, let me know if you are still interested”, I wanted to avoid any spam reports.
Most people replied back and wanted to test my tool, one replied he doesn’t remember what it’s about, I wished him a good day, some didn’t, I didn’t follow up.
The people who wanted to test the tool, I send them to my site to register and then come back with their email so I can open their user like they actually paid and there are no limits, in retro spec that was very time consuming, and I should have automatically enabled all registration to be like they paid for the beta.
During my communication with the testers some gave me immediate feedback, some asked questions:
- What’s the difference between my product and product X?
- Which geography does my keyword database supports?
- General information about how I did the product
Some gave me feedback that was not related to the testing but gave me valuable information:
- Marketing avenues I didn’t consider
- Information about who uses the competition and how
- One tester told me he found a critical issue in his site, that my tool uncovered
One thing that people asked for and I wasn’t expecting is if I have a forum/system to report the bugs, I told them that they can reply to me, but the fact that about five testers asked it, was a surprise to me.
I think the beta went well, I discovered new things about myself and about my site:
I come from the world of B2B, and I’m not used to talking to so many people in one day, I felt just a little the mass reach of B2C. I had this buzzing energy, first time I felt it, it’s an energy that makes you want to run or do something energetic, not sit down or relax.
Also, when I visited the profiles of people who replied to my Facebook message, and the common theme in most profiles was pictures of happiness, being with friends, or family. Which strengthen one of my core beliefs that people just want to be happy.
Summary of the test
- Total duration was 3 days
- I got 71 registrations to test the product, from all around the world
- I got 1000 unique page views
- Nothing crashed
- About ten people added me as Facebook friend
- Most people used Gmail for the registration email
More detailed breakdown (Some of the Israel visits are probably mine)
What did I learn about my product?
- Everything worked, there were very few bugs, and I was able to resolve them quickly
- Server’s load was very low, so I have some time before I worry about it
- People mostly liked the product
- The feedback that was repeated more than once is the size of the backlinks database, which should grow over the next few months
- People liked the design and look
- There were little graphic issues, like bad rendering across different devices
- I saw how my analytics work, and found where I need to strengthen it
- Some of the features I thought people would use, I didn’t get any feedback on it
- Be open to feedback, I did hear some things I didn’t agree with, but I always thanked the tester
- Make sure your emails don’t land in Gmail spam folder
Now that the beta is over, I need to finish the last loose ends I have with the site and start the thing that is a challenge for me – marketing it.
Overall people were very helpful and positive, and I was surprised for good by the entire process.
I did discover that after the initial conversation none of the testers came back to me with feedback, I decided not to reach out, I got most of the input I wanted. Also the product didn’t stick, which means none stayed and use it, although they had full account.
Recap of what I learnt
Summary of things I learnt:
- Facebook limits the number of outgoing messages, ask people to message you.
- Make it easier for people to register with the test account.
- There’s a message request tab in Facebook messenger, and there’s a “see filtered result” option in it too.
- Most people are positive and happy to help.
- Always listen to your potential clients.
- People want to be happy.
- You need to follow up to get feedback.