is happily being the wheel rather than a rusty old spoke
Register for free to join our community of investors and share your ideas. You will also get access to streaming quotes, interactive charts, trades, portfolio, live options flow and more tools.
Register for free to join our community of investors and share your ideas. You will also get access to streaming quotes, interactive charts, trades, portfolio, live options flow and more tools.
test
Yep. Coincidence. I've been taming a lot of over-active routines and queries today, just in read_msg.asp, which is by far the most-used routine on the site.
Getting better all the time. I set a goal for myself, which I've already exceeded, but am still plugging away.
Finding small leaks and inefficiences that aren't much of anything on their own, but apparently have been adding up to a lot. Most recent example is the routine that, while you're reading a message, updates the count used to determine how many unread messages there are by a person on your Favorites page. I was calling that routine regardless of whether a person was logged in, which wasn't resulting in an error, but was resulting in wasted computing power. In the 10-second range I observed, it looks like it was getting called about 70% of the time by non-logins. Now it's only called if you're logged in.
I doubt it's much of a load (free members do get plenty of posts), but should be easy enough to check to see if anyone has a post count for a day in excess of their allowed posts and do whatever needs to be done about it.
Hopefully Dave or Matt will look into your particular situation when they get a chance to see if I broke something. At the moment, I'm still pretty deep into tweaking read_msg.asp
test
There was a problem with certain (most?) scenarios of reading the last message in a board not resetting the count on Favorites. Fixed.
Looks plausible. It's when reads are more than 100 times posts that I start getting suspicious.
A change I just put into production was to remove what looked like a completely extraneous block of code dealing with Ignores.
Let me know if there's anything weird happening in message-reading, particularly where Ignores are concerned.
IIS5.0 ASP (VBScript) and SQL Server 2000 backend.
It'll soon become SQL Server 2005 backend on a much more powerful machine, and a biggie project I hope to get started in the next couple of months is rewriting the front-end to IIS6/ASP.NET.
I decided that for now, rather than rewriting the posting/reading routines from scratch to be as efficient as possible, I'm going with the more expedient route of modifying existing routines to extract more efficiency out of them and saving the rewrites for when the status quo is finally acceptable.
That means I'm going to be implementing new versions of each minor change throughout the day. Please let me know if you see any problems. I'm currently working on read_msg and have implemented some fairly small changes.
I'll at least test the Premium user scenario before implementing. If you and Matt are available to test the free and non-logged in scenarios, that'd be beautiful.
You really shouldn't have to.
Read_msg.asp should reset it when you reach the last message. It's working that way for me now.
What I've got as Priority One is not only for the sake of overall system availability but for the sake of getting search as near real-time as possible. We've found that we simply can't have search updating happening right now as it adds so much workload to the db server, it can't adequately handle everything else.
So the idea here is to relieve the other burdens so it can start handling search again. It'll probably take a few days to finish the most important/needed tweaks, then try making search realtime again. If that still doesn't work, I've got another idea up my sleeve (moving search to SI's db) that should work provided I don't run into any barriers when implementing it, but it'll take a while to implement.
No, you don't need to log out and back in.
I'm not sure why Favorites wouldn't be resetting. Although there's the possibility that a post is getting written in the time it takes for you to read the last message and go back to Favorites.
Or are you seeing it consistently?
It's working correctly for me.
The horses are already out of the barn. New traffic brings more load, but it's miniscule compared to the existing load.
I guess you're human after all!
You take that back!!! LOL
I may bitch and moan but at least with IHUB the powers that be are reachable.
With other websites it's like spitting in the wind.
I think it's as a friend was telling me yesterday. We're unique in that all the "powers that be" on our sites are hard-core message-board guys. It's our (the sites') biggest strength. Unfortunately, though we're far from technically incompetent, we're not the most exceptional geeks around. Though in my case, having worked for years as a consultant, I take this mis-step particularly personally, but don't worry. I won't take it out on you and I haven't even been kicking the dogs. Any more than usual. <g>
I just have to hunker down and fix it. And I'm confident I will. Anytime the equipment and their workload back me into a corner, I've ALWAYS found some more performance in the couch cushions. Which is why we've got a Load Balancer sitting in the office that we bought about 2 years ago when I thought it and multiple webservers were our only solution, only to find a ton of wasted performance to free up while waiting for the LB to arrive.
The LB and multiple webservers aren't even a "potential" solution right now. The webserver, while working hard, is not the bottleneck. It's the database server. And I'm working on getting it fixed.
And I'm going to be breaking a huge rule of mine. I'm so drained at night lately that I'm getting zero programming/tweaking done at night, so I'm going to be making changes to important routines and implementing them mid-day.
But with the new way I publish code, if it's discovered that I've broken something, a rollback is easy. Can roll-back up to the 10 most recent versions of any program.
Edit: That Submit took a little while, but didn't time out. Histograms are going way up. They're really displaying an unnatural pattern that makes me inclined to look closely at the log files before I do anything else.
Fixed. I broke it yesterday when tweaking go.asp
Priority one is to increase system availability (reduce slowdowns) and priority two is giving full-text search capability in some fashion since it's a biggie Premium feature. And both of these have to be accomplished soon because though the new box is ordered, it won't be here for 2-3 weeks and then it'll take a while to configure/install it. And hopefully the new webserver along with it, which has just been sitting there idling since the install.
Edit: So far, so good. I'm pleased with the responsiveness of the system right now, during one of the busiest half hours of the day. Though I'm on dialup so "normal" for me might be pathetic for broadband users, which I'll be about 10 minutes from now.
Knew it. Right around 9:20, traffic and CPU utilization start their serious ramp-up.
Fixed the problem with Favorites counts not resetting after reading a board.
test
test
Damn! I KNEW that proc I edited and used for go.asp (what gets used when you hit the new post count from the Favorites page) couldn't just be a forgotten and unused one.
I'll fix it now. I'm sure what you're seeing is that when you get to the last post in a thread, it's not resetting like it used to if someone posted between your hitting the number on Favorites and your reaching the last message.
Churak, I'm closely watching the histogram to see what happens at 9:20.
test
test
I wouldn't be, so I wouldn't expect others to be. I simply didn't leave myself the kind of margin I should've. Lesson learned.
test
test
test
Not so eerie when you consider the dynamics of a very large group of people. The group itself, because of its sheer size, has a very distinct pattern that's more predictable than just observing a few people.
Failed "Submit" attempts. The number of replies gets incremented pretty early in the process. Before the part that's most prone to failure. I need to switch the order in which those things happen to reduce the indicence of that problem.
That's REAL bad when two posts 6 seconds apart got the same number.
Not surprising at all. That's when the traffic load starts to come on in full force.
The new server will definitely help and will make these problems go away completely (for now), especially when we also finally put the new web server into production. But there's still a lot of programming to be done to make sure we're ready to handle another 50% increase in traffic when it happens.
I really got caught flat-footed on this one. I knew that due to organic growth, I needed to do this later this year, but the non-organic growth highlighted that I should've done it long ago so we'd be ready.
Right now, even without the load of full-text indexing, the machines are taking a real beating since it's the busiest part of the trading day.
The search function is still intact, but we're finding that on the current setup, we simply can't afford to have the system try to maintain the indexes (the mechanism through which posts containing keywords are found) in realtime. So I've temporarily disabled the population of those indexes and will restart them every night and shut them off every morning.
The new database server will be ordered tomorrow and while I'm waiting for it to arrive and be installed I've got a few things to work on to improve efficiency and possibly make realtime indexing possible again either by reducing other workloads on the db server, or moving the indexing/search function to SI's db server, which has plenty of capacity for it.
test
Thanks for confirming my suspicion. I was searching for something else and found a post in this thread, and couldn't believe it when I saw evidence this used to be HCCA.
It was being heavily promoted on SI years ago.
Have the finally decided to focus only on pretending to be into mining? Or do they still pretend to be into healthcare and recording studios? I can still remember when it did something like a 1000:1 reverse split.
I really can't believe I'm still seeing the name "Furlong" associated with a public company. Anthony Elgindy sits in jail for mostly helping people benefit from identifying scams, yet Furlong is still running free and keeping up his game?
There's simply no justice.
Yes, free members are especially prone to slowness. Our own ad server isn't much of a bottleneck, despite dealing with about 4 million or so hits per day (it serves both iHub and SI), but when we're not serving an ad from it, it has to make a call to our top-tier network and if they've seen you enough times to use up the ads they want to show you, they pass you to another network, and so on. Through about 8 of them. As the day progresses, a free user can really start seeing a slowdown. And gradually worse ads.
When you start seeing ads offering free xBox's or iPods, you've been on the site a while today. <g>
Geez! It's really crawling from my home computer, which is dialed in to the same ISP the site's hosted from! Oh, this is nuts! It starts out in KC, goes to St. Louis, then Chicago, then a dozen parts unknown, then back to our webserver, which I know is located about 20 feet away from the rack I'm dialed in to.
Anyway, the only "better solution" I really can think of is lots of memory. It's cheap, but not everybody knows how to add it.
My office computer has 1.5GB of memory. Back when that was a ton. Bought it in 2002. Actually, I think it's one of the first iHub boxes I bought or I bought it at the same time.
I used to occasionally get errors that I didn't have enough memory to open an app, but I don't think that's ever happened since upgrading to XP. My laptop has a gig of memory and it's never had problems. XP on this one, too.
I'm really puzzled. I mean, it looks like the whole path from your place to our webserver is dog-slow with the exception of our webserver itself!