Thoughts and Comments on the Recent Downtime

Your home for general discussion. Don't know where to post? Try here first.
Post Reply
User avatar
centrop67
Site Admin
Site Admin
Posts: 3581
Joined: Mon Dec 03, 2012 3:42 pm
Tell us about yourself: Owned by Hondo and Mustang - two MAD rescue dogs.
Location: Cutler Bay, FL, USA

Thoughts and Comments on the Recent Downtime

Post by centrop67 »

After nearly 6 days of downtime the forum website is again active. I apologize for any inconvenience this may have caused.

Some things to consider...
1. No one is making money off this site, so there was no financial impact.
2. There is no life or death need to have this site. I make it available for entertainment and knowledge sharing.

Having said that, I am interested in hearing everyone's opinion on this. Please share your thoughts and feelings in this thread.

Thanks.
Michael
Location - Cutler Bay, FL USA
Image RIP: Leela, Radar, Tequila, and Snickers
If there are no dogs in Heaven, then when I die I want to go where they went. - Will Rogers
TimL_168
Working Dog
Posts: 1371
Joined: Wed Oct 08, 2014 2:11 pm
Tell us about yourself: I am: a father of 2 boys, a carpenter, hunter, runner. We have extensive experience with sled dogs, shepherd mixes, a wolf hybrid, and our current dog a 95# long haired Shiloh Shepherd. We added Endeavor in April 2016. She was not working out in HRD. I train for game recovery and general utility.
Location: central MD

Re: Thoughts and Comments on the Recent Downtime

Post by TimL_168 »

Frustrating. We've got a nice community here
Tim L.
Aurora(Shiloh) Endeavor
User avatar
centrop67
Site Admin
Site Admin
Posts: 3581
Joined: Mon Dec 03, 2012 3:42 pm
Tell us about yourself: Owned by Hondo and Mustang - two MAD rescue dogs.
Location: Cutler Bay, FL, USA

Re: Thoughts and Comments on the Recent Downtime

Post by centrop67 »

Thanks Tim. I hear you, and I feel the same way.
Michael
Location - Cutler Bay, FL USA
Image RIP: Leela, Radar, Tequila, and Snickers
If there are no dogs in Heaven, then when I die I want to go where they went. - Will Rogers
Mark77
Training Dog
Posts: 468
Joined: Fri Aug 03, 2012 6:19 am
Tell us about yourself: Own a Mal and a Dutchie.

Re: Thoughts and Comments on the Recent Downtime

Post by Mark77 »

What was the reason for the site being down?
Joxgirl
Working Dog
Posts: 1634
Joined: Wed Apr 26, 2017 1:38 pm
Tell us about yourself: Loving every crazy day with my DS in CA and UT.
Location: Southern California

Re: Thoughts and Comments on the Recent Downtime

Post by Joxgirl »

You had me in a panic. I was looking for it and had trouble finding it. I felt bad for being absent so long and hoped that it was gone forever.
Then Steve asked what happened to it today and I felt panicked again thinking that maybe it was no longer active.
My shortcut even disappeared.
What happened?
Saralee
Rogue (Dutch Shepherd) 2 years old
Joachimstaler @ IG
User avatar
centrop67
Site Admin
Site Admin
Posts: 3581
Joined: Mon Dec 03, 2012 3:42 pm
Tell us about yourself: Owned by Hondo and Mustang - two MAD rescue dogs.
Location: Cutler Bay, FL, USA

Re: Thoughts and Comments on the Recent Downtime

Post by centrop67 »

At around 5:30pm EDT last Friday (9/21/18) the hosting company where the forum website resides suffered a major outage.

Image

The site eventually came back up early Wednesday (9/26/18) morning.

The hosting company admits that a set of multiple mistakes, assumptions, and damaged backup data led them to the problem and hassle bringing their system back up. Here's a snippet from the communication sent during the outage:
Michael Denney - MDDHosting LLC wrote:**What happened?**

First and foremost - this failure is not something that we planned on or expected. A server administrator, the most experienced administrator we have, made a big mistake. During some routine maintenance where they were supposed to perform a _file system trim_ they mistakenly performed a _block discard_.

**What does this mean?**

The server administrator essentially told our storage platform to drop all data rather than simply dropping data that had been marked as _deleted_ by our servers.

**Why is restoration taking so long?**

Initially we believed that only the primary operating system partition of the servers was damaged - so we worked to bring new machines online to connect to our storage to bring accounts back online. Had our initial belief been correct - we'd have been back online in a few hours at most.

As it turns out our local data was corrupted beyond repair - to the point that we could not even mount the file systems to attempt data recovery.
Michael
Location - Cutler Bay, FL USA
Image RIP: Leela, Radar, Tequila, and Snickers
If there are no dogs in Heaven, then when I die I want to go where they went. - Will Rogers
User avatar
centrop67
Site Admin
Site Admin
Posts: 3581
Joined: Mon Dec 03, 2012 3:42 pm
Tell us about yourself: Owned by Hondo and Mustang - two MAD rescue dogs.
Location: Cutler Bay, FL, USA

Re: Thoughts and Comments on the Recent Downtime

Post by centrop67 »

As an FYI - I do promote the website with a Facebook page, and while I know some of you do not do Facebook or any social media, I kept posting updates during the problem on Facebook.

I am willing to expand to other social media tools such as Twitter or Instagram if there are members not on FB but on these platforms.

Please feel free to share if you have a preferred platform, and I will start promoting the website there as well.
Michael
Location - Cutler Bay, FL USA
Image RIP: Leela, Radar, Tequila, and Snickers
If there are no dogs in Heaven, then when I die I want to go where they went. - Will Rogers
User avatar
centrop67
Site Admin
Site Admin
Posts: 3581
Joined: Mon Dec 03, 2012 3:42 pm
Tell us about yourself: Owned by Hondo and Mustang - two MAD rescue dogs.
Location: Cutler Bay, FL, USA

Re: Thoughts and Comments on the Recent Downtime

Post by centrop67 »

Joxgirl wrote: Thu Sep 27, 2018 2:15 am I felt bad for being absent so long and hoped that it was gone forever.
:lol:

Saralee, I am hoping you meant to write "wasn't gone forever."

I need to start thinking about the future of the site. Right now, if something happens to me, there is no contingency for keeping it going. :|
Michael
Location - Cutler Bay, FL USA
Image RIP: Leela, Radar, Tequila, and Snickers
If there are no dogs in Heaven, then when I die I want to go where they went. - Will Rogers
User avatar
Tennessee3
Green Dog
Posts: 139
Joined: Wed Aug 16, 2017 8:29 pm
Tell us about yourself: Nicholas or Nick, doesn't matter to me.

Re: Thoughts and Comments on the Recent Downtime

Post by Tennessee3 »

I’m still coming to terms with it, it’s just too soon for me to talk about what happened.

I had to go on Facebook to find out what was going on.... I feel dirty just typing that.
Nicholas

Millie WLGSD in training for Schutzhund 5/27/17 & Grendel KNPV lines Malinois same + PP work 2/15/18

...... Mals are tan dutchies, right?

https://www.instagram.com/nick_millie_and_grendel/
User avatar
borellar15
Training Dog
Posts: 390
Joined: Sun Nov 26, 2017 11:28 pm
Tell us about yourself: Living in Austin, Texas with our dogs Zoey(DS), Henry (DS) and Tripper(labX). In college, I worked as a Handler and Assistant Manager of a training and daycare facility for 3 years where I've handled many working dogs. For the past 2 years I have been training Zoey and Henry and we have made some tremendous progress together.
Location: Austin, Texas

Re: Thoughts and Comments on the Recent Downtime

Post by borellar15 »

Tennessee3 wrote: Fri Sep 28, 2018 5:11 pm I’m still coming to terms with it, it’s just too soon for me to talk about what happened.
:lol:
Image Image
Alex - Zoey(DS), Henry (DS), Tripper Lab mix. Austin, Texas
User avatar
Tennessee3
Green Dog
Posts: 139
Joined: Wed Aug 16, 2017 8:29 pm
Tell us about yourself: Nicholas or Nick, doesn't matter to me.

Re: Thoughts and Comments on the Recent Downtime

Post by Tennessee3 »

centrop67 wrote: Thu Sep 27, 2018 11:54 am
Joxgirl wrote: Thu Sep 27, 2018 2:15 am I felt bad for being absent so long and hoped that it was gone forever.
:lol:

Saralee, I am hoping you meant to write "wasn't gone forever."

I need to start thinking about the future of the site. Right now, if something happens to me, there is no contingency for keeping it going. :|
I’ll take over as admin, I’ll forward a list of my terms Monday morning.

Wednesday is looking strong for a contingency planning meeting.
Nicholas

Millie WLGSD in training for Schutzhund 5/27/17 & Grendel KNPV lines Malinois same + PP work 2/15/18

...... Mals are tan dutchies, right?

https://www.instagram.com/nick_millie_and_grendel/
User avatar
Tennessee3
Green Dog
Posts: 139
Joined: Wed Aug 16, 2017 8:29 pm
Tell us about yourself: Nicholas or Nick, doesn't matter to me.

Re: Thoughts and Comments on the Recent Downtime

Post by Tennessee3 »

Sorry Cent. I know this is a big deal and I’m making jokes.

It was annoying, this is one of the few places left that has a good membership of people that I can share with & learn from concerning working dogs like these. And provides a great resource and an ethical promotion of the breed.
Nicholas

Millie WLGSD in training for Schutzhund 5/27/17 & Grendel KNPV lines Malinois same + PP work 2/15/18

...... Mals are tan dutchies, right?

https://www.instagram.com/nick_millie_and_grendel/
Post Reply