Previously, our databases and application primarily operated out of our co-location hosting facility in Los Angeles, CA, with a warm backup facility in Seattle, WA. While the facilities provided by 365 Main were extremely high quality, the physical equipment made it difficult to scale with our rapidly growing company. It honestly became difficult to deploy new servers fast enough to keep up with our growth.
In November 2010, we started migrating our tracking servers to AWS, and in January 2011 we launched our globally load-balanced ad servers. Over the past year, we’ve been learning more and more about AWS and how to build and scale platforms on their infrastructure as there were some interesting new challenges. We started deploying app servers on AWS for our Dedicated Solutions customers in June of 2011, and since that time we began improving our deployment of app servers on AWS.
On December 14, 2011, we started migrating all our hosted clients’ applications to AWS. It was a two step process, first replicate the data from the co-lo facility to the main AWS facility – USE – in Virginia, United States (East). Then setup API servers for the data layer and launch application servers.
We can now deploy a new API or application server in under thirty minutes. And we don’t need to wait for the server to be built or installed in a server rack. While many say AWS is the most expensive solution, AWS allows us to focus on building and deploying the best technology – allowing us to be a true software company instead of a software and hardware company.
Three Times More Servers
These new deployments on AWS further segregated our clients, allowing us to group them by edition for sharing server resources. Dedicated Solutions clients receive their own deployment, not sharing resources with any other clients.
We’ve deployed three times more API and application servers on AWS than the number we had in our co-lo facilities. This increased capacity vastly improves performance for our current clients, providing substantial capacity for our current clients to grow in their current application and API pools.
Each deployment consists of at least two application servers, two API servers, two memcache servers, two master database servers, and two slave database servers. The application, API and memcache servers all utilize AWS Large Instances (average cost of $300~ per month). The database servers use AWS High-Memory Extra Large Instance (average cost of $375~ per month). You can learn more about these servers here:
http://aws.amazon.com/ec2/
The new deployment also allows our system admins to easily migrate clients from one application or API pool to another or even move the client to their own Dedicated Solution.
Instead of assigning clients to the application, API and database pools with a round robin methodology, new clients are only assigned to the last application, API and database pools. Then when this pool reaches capacity, HasOffers creates a new deployment of new application, API and database servers. This isolates our current clients in server pools that are not interrupted by new HasOffers clients, ensuring optimal performance. This new account/client assignment is a system we have wanted to deploy for many months, but the limitations of physical systems prevented us from having the capacity necessary.
Dedicated API Servers
With this cloud deployment, API V2 is now deployed on separate dedicated servers. This allows us to scale external use of the API V2 independently from application use.
Now, if you include your network ID in the domain for API requests, HasOffers will direct your requests to your specific API servers.
http://networkid.api.
API requests for Enterprise clients are still limited. Currently the limit is set to 40 requests in 10 seconds. You can contact your account manager for pricing on increased API rate limits.
With the migration of the HasOffers app and API to AWS, we wanted to provide more transparency for our clients. While we have several internal and external monitors of all our systems, we’ve setup a publicly available monitor page using Pingdom.com here:
The monitors on the status page check the defined systems once a minute. If the systems have any issues, the status page is updated in real-time. Each system check not only checks to see if the systems are online but actually returns back a specific piece of data depending on the system being checked. If you ever perceive an issue with availability or performance, please consult this page to confirm that a known issue is taking place and what system is being impacted. Please rest assured that any issue recorded here is currently under investigation by HasOffers engineers.
For the last two years, HasOffers has used Zendesk to manage support requests while our support documentation was located on the HasOffers Wiki. This month we migrated all of our documentation from the wiki to Zendesk, making it easier for our teams to make updates and for all HasOffers support related materials to be found in one place.
By integrating the documentation with our primary support tool Zendesk, we hope to provide a more holistic support experience. If you don’t have a user account for our Zendesk support system, feel free to create one at any time by visiting support.hasoffers.com.
With your own account on the HasOffers support system you can:
Rate documentation articles
Comment on forum articles
Submit feature requests
Submit support tickets
View and follow open support tickets
New Support Team Members
Over the last two months HasOffers has added three new technical support specialists to continue improving upon technical support for our clients. You may have met David or Justin recently, and now Jeff has also joined our team to cover a 11pm until 7am shift.
Looking Forward
We are now in an incredible position for 2012. Increased performance and capacity allows us to concentrate more on improving efficiencies of the application and building new features and integrations. We already have an exciting list of new innovations provided by your feedback, so please keep the suggestions coming. Thank you so much for your patience in this time of transition and we look forward to supporting your business in the coming year.
