We have narrowed the problem down to a TCP/IP connection issue between our web servers and the Internet. Our engineering team is making good progress in isolating the root cause and is performing additional tests over the next 24 hours. As a result, we hope to validate a solution to restore normal performance of the SmartVault service. We will update our customers tomorrow afternoon with our progress.
Status Report: We have narrowed the problem down to a TCP/IP connection issue in the Windows OS of our web servers. Yesterday, a Windows server upgrade appeared to fix the problem in testing, but a trial this afternoon in our production network showed the same recurring problem while under heavy load. Our engineering team is performing additional diagnostics to get to the root cause of the problem. We will be updating our customers daily with progress updates.
Background: On Tuesday, November 1st, we performed a major maintenance upgrade to our data center. “Major” means that this is an upgrade that is typically done only once every two to three years. We scheduled this maintenance release to occur as soon as possible after tax extension season, and well ahead of the upcoming tax season, to have minimal risk to the thousands of accounting professionals that depend on SmartVault.
On Wednesday, November 2nd, as a result of changes introduced in the maintenance release, we experienced a partial service outage during periods of heavy usage. We restored service, but began experiencing performance issues in some areas of the SmartVault service during peak usage hours between 11am – 3pm central time. Working through the weekend, our engineering team has been able to identify the potential root causes for the degraded service, and are working to implement changes needed to bring the performance back to normal at peak hours.
Work-Arounds for Customers: To minimize this disruption to customers, there are 3 workarounds that should increase SmartVault’s performance in your environment:
We sincerely apologize for the inconvenience and appreciate your support as we work on fixing this performance issue. Our engineering team is working hard to pinpoint the root cause of the issue so that service is restored to peak performance as soon as possible. We will post updates to this page daily.
713 479 5401 direct