I'm interested in the idea that the auction house should have been tested more before going live. Like, how long should it have stayed on the PTR? Until millions of people all posted probably billions of items within a few hours of each other one day? Because that's the test.
-
-
It’s not always possible to simulate an actual live environment for testing on the scale you would expect
-
With 15 years of data from the current AH, it should be fairly straight forward to simulate a live enviroment.
- Show replies
New conversation -
-
-
This. So. Much. It boggles my mind that people think that all testing needs to be done by people. Automation is a thing. Especially for stress testing things like the AH.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
What
@TaliesinEvitel talked about is conditions that only integration testing will catch, not unit testing, still, there arre ways to automate that as well, run the battery of tests before build, catch regressions etc. -
Yeah. Not having that lot done... It's certainly interesting. Again, going on what I've heard, it's a real hard time (re deadlines) over there.
- Show replies
New conversation -
-
-
(not a gotcha ofc, goal is to share experience & le knowledge - if you're ever interested in that kinda thing & can go into more depth / can pass on questions to people who have more specified knowledge than I do)
-
I asked the question :)
- Show replies
New conversation -
-
-
I hear you, but I would also posit that no MMOG manages to go live on stuff like this problem-free, which suggests it may not actually be possible to identify some problems outside of a live environment with millions of real interactions (1/2)
-
(2/2) and if that's the case (as the evidence suggests) then we should probably judge dev teams on how quickly they identify and fix those live issues, than on whether those issues go live in the first place.
- Show replies
New conversation -
-
-
Yes, test engineers are invaluable to a project. But it's easy to say that they didn't test a system (enough), if you don't know the underlying issue. Unit tests can catch errors early in the dev cycle. But they won't catch everything, especially those in distributed systems. 1/2
-
Simulations and integration/system tests will cover those areas, but they won't catch every major issue. Lastly, while your testing envs are as similar to your production envs, there will most likely be differences, that may cause new issues. 2/2
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.