November 10, 2015
A little while ago, we promised a post about our development roadmap, as well as what has been worked on in the last year. There is a laundry list of excellent features and updates that have been waiting in our pipeline for what feels like forever. And understandably there’s a lot of questioning on the forums over why they haven’t materialized yet.
Search improvements, ratings improvements, a shopping cart, item support tools, affiliate campaigns and analytics, license improvements and the list goes on. It’s a list of things that would make buying better, selling better, and alleviate problems left, right and center.
The long and the short is that a lot of these types of improvements keep getting put on hold as we cope with growth and expansion.
Growth and Expansion
The growth on our marketplaces has been breakneck over the last few years with everything from revenue to user count to transaction volume to item count to fraud attempts to security hacks doubling, tripling, quadrupling or more year on year. To be honest I find a lot of the numbers kind of staggering.
On the scaling front, I checked in with our lead architect Pete who tells me we now have 8 physical machines with 68 cores and 432 GB of RAM , plus 17 virtual machines including 9 front-end servers, and we are serving up some 150 TB of traffic a month. In short, it’s a truckload of horsepower, and just scaling it all is a huge load of work. But it’s paid off, despite the enormous growth in traffic, you will notice that our sites are pretty snappy and rarely run into downtime. Security and fraud also take up quite a bit of time, and the bigger we get the more of an issue they become. Our level of response sophistication keeps rising as the threat level does. We are coping with both the scaling and security and fraud challenges, but it is a large amount of development work every month!