Page 1 of 1
Thoughts and Comments on the Recent Downtime
Posted: Wed Sep 26, 2018 11:47 am
by centrop67
After nearly 6 days of downtime the forum website is again active. I apologize for any inconvenience this may have caused.
Some things to consider...
1. No one is making money off this site, so there was no financial impact.
2. There is no life or death need to have this site. I make it available for entertainment and knowledge sharing.
Having said that, I am interested in hearing everyone's opinion on this. Please share your thoughts and feelings in this thread.
Thanks.
Re: Thoughts and Comments on the Recent Downtime
Posted: Wed Sep 26, 2018 12:04 pm
by TimL_168
Frustrating. We've got a nice community here
Re: Thoughts and Comments on the Recent Downtime
Posted: Wed Sep 26, 2018 12:06 pm
by centrop67
Thanks Tim. I hear you, and I feel the same way.
Re: Thoughts and Comments on the Recent Downtime
Posted: Thu Sep 27, 2018 12:41 am
by Mark77
What was the reason for the site being down?
Re: Thoughts and Comments on the Recent Downtime
Posted: Thu Sep 27, 2018 2:15 am
by Joxgirl
You had me in a panic. I was looking for it and had trouble finding it. I felt bad for being absent so long and hoped that it was gone forever.
Then Steve asked what happened to it today and I felt panicked again thinking that maybe it was no longer active.
My shortcut even disappeared.
What happened?
Re: Thoughts and Comments on the Recent Downtime
Posted: Thu Sep 27, 2018 11:43 am
by centrop67
At around 5:30pm EDT last Friday (9/21/18) the hosting company where the forum website resides suffered a major outage.
The site eventually came back up early Wednesday (9/26/18) morning.
The hosting company admits that a set of multiple mistakes, assumptions, and damaged backup data led them to the problem and hassle bringing their system back up. Here's a snippet from the communication sent during the outage:
Michael Denney - MDDHosting LLC wrote:**What happened?**
First and foremost - this failure is not something that we planned on or expected. A server administrator, the most experienced administrator we have, made a big mistake. During some routine maintenance where they were supposed to perform a _file system trim_ they mistakenly performed a _block discard_.
**What does this mean?**
The server administrator essentially told our storage platform to drop all data rather than simply dropping data that had been marked as _deleted_ by our servers.
**Why is restoration taking so long?**
Initially we believed that only the primary operating system partition of the servers was damaged - so we worked to bring new machines online to connect to our storage to bring accounts back online. Had our initial belief been correct - we'd have been back online in a few hours at most.
As it turns out our local data was corrupted beyond repair - to the point that we could not even mount the file systems to attempt data recovery.
Re: Thoughts and Comments on the Recent Downtime
Posted: Thu Sep 27, 2018 11:48 am
by centrop67
As an FYI - I do promote the website with a Facebook page, and while I know some of you do not do Facebook or any social media, I kept posting updates during the problem on Facebook.
I am willing to expand to other social media tools such as Twitter or Instagram if there are members not on FB but on these platforms.
Please feel free to share if you have a preferred platform, and I will start promoting the website there as well.
Re: Thoughts and Comments on the Recent Downtime
Posted: Thu Sep 27, 2018 11:54 am
by centrop67
Joxgirl wrote: ↑Thu Sep 27, 2018 2:15 am
I felt bad for being absent so long and hoped that it
was gone forever.
Saralee, I am hoping you meant to write "
wasn't gone forever."
I need to start thinking about the future of the site. Right now, if something happens to me, there is no contingency for keeping it going.
Re: Thoughts and Comments on the Recent Downtime
Posted: Fri Sep 28, 2018 5:11 pm
by Tennessee3
I’m still coming to terms with it, it’s just too soon for me to talk about what happened.
I had to go on Facebook to find out what was going on.... I feel dirty just typing that.
Re: Thoughts and Comments on the Recent Downtime
Posted: Fri Sep 28, 2018 7:38 pm
by borellar15
Tennessee3 wrote: ↑Fri Sep 28, 2018 5:11 pm
I’m still coming to terms with it, it’s just too soon for me to talk about what happened.
Re: Thoughts and Comments on the Recent Downtime
Posted: Sat Sep 29, 2018 2:45 am
by Tennessee3
centrop67 wrote: ↑Thu Sep 27, 2018 11:54 am
Joxgirl wrote: ↑Thu Sep 27, 2018 2:15 am
I felt bad for being absent so long and hoped that it
was gone forever.
Saralee, I am hoping you meant to write "
wasn't gone forever."
I need to start thinking about the future of the site. Right now, if something happens to me, there is no contingency for keeping it going.
I’ll take over as admin, I’ll forward a list of my terms Monday morning.
Wednesday is looking strong for a contingency planning meeting.
Re: Thoughts and Comments on the Recent Downtime
Posted: Sat Sep 29, 2018 8:12 pm
by Tennessee3
Sorry Cent. I know this is a big deal and I’m making jokes.
It was annoying, this is one of the few places left that has a good membership of people that I can share with & learn from concerning working dogs like these. And provides a great resource and an ethical promotion of the breed.