[Faccus] Update 8:55am: Status of LEARN - site unavailable after problems discovered

chappell at uwaterloo.ca chappell at uwaterloo.ca
Thu Jan 31 09:00:59 EST 2013

[Thu., Jan. 31, 8:55am]
Hello again,
We reported to D2L the HTTP 500 errors and also that our system was very
slow. They know there is a problem and were working on it. We agreed it
was better to have the system unavailable than run with those problems. We
do not have a recovery time as yet, sorry.

> [Thu., Jan. 31, D2L 7:45am update]
> Hello,
> Desire2Learn made our system available again and said we can "assume that
> all work has been completed and that we have finished our verification
> testing of your site". However, the support team is seeing errors (HTTP
> 500) and has had reports of problems logging in by students and faculty
> members. The system is very slow. We will be talking to D2L about this and
> we will keep you updated as soon as we know more.
> Andrea
>> [Thu., Jan. 31, D2L 4:25am update]
>> Hello,
>> D2L sent an update at midnight, indicating that the process they had
>> anticipated finishing by then was 98% done, and that once it completed,
>> they would start verification.
>> At 4:25am the update indicates a 2-3 hour process needed. It is a bit
>> confusing as to what this process is, except that it involves migration
>> of
>> our data and files to the new configuration. They will provide a further
>> status update upon completion, which could be 6:30 - 7:30am based on
>> their
>> estimate.
>> Andrea
>>> [Wed. Jan. 30, 10:00pm update]
>>> Hello,
>>> The latest message from D2L, at 10:12pm, says they still expect to have
>>> the recovery by midnight. They have mentioned the need for verification
>>> but have not indicated how long that might take. They have said they'll
>>> take off the "maintenance" message when the recovery and verification
>>> is
>>> complete. I assume that means they will remove the redirect then.
>>> Questions are arising individually about what will happen with
>>> assignments
>>> and tests. I think instructors are altering due dates for assignments,
>>> but
>>> mid-terms are likely more difficult. Hopefully instructors will post
>>> announcements in their courses to indicate any changes, so student see
>>> those as the system is back up.
>>> Next update is at midnight and I may not be awake for that!
>>> Andrea
>>>> [Wed. Jan. 30, 5:45pm update]
>>>> Hello,
>>>> The changes anticipated to be complete by 6:00pm are not complete. The
>>> revised finish time is estimated for between 10:00pm and midnight,
>>> followed by a verification process.
>>>> The Waterloo LEARN system remains inaccessible during this time.
>>>> Access
>>> to
>>>> the barely responsive system created many more problems for our
>>>> faculty
>>> and students (long wait times to access the system, if at all,
>>>> interruptions to most course functions making those functions
>>>> unusable,
>>> inability to access files, etc.).
>>>> We expect the next update at about 10pm. At this time the system has
>>> for
>>>> 27 hours. We will be seeking detailed explanations from D2L, as well
>>>> as
>>> their actions to prevent this from happening again.
>>>> Andrea
>>>>> [Wed. Jan. 30, 3:30pm update]
>>>>> Hello,
>>>>> The message from D2L remains the same, that the expected up-time is
>>> 6pm.
>>>> Thanks to Pat Lafranier for enlisting the help of Jonathan Woodcock to
>>>> put
>>>>> for now a "Help with LEARN" button on the pathways pages on the
>>> Waterloo
>>>> website, to point to our LEARN update page, maintained by Daspina
>>>> Fefekos.
>>>>> https://uwaterloo.ca/learn-help/
>>>>> Our next update from D2L is expected at 6pm and I will send a message
>>>> again then.
>>>>> Andrea
>>>>>> [Wed. Jan. 30, 12:30pm update]
>>>>>> (Apologies if you receive a second copy of this message. There
>>> appeared
>>>>> to
>>>>>> be a problem when I sent it the first time, and I want to make sure
>>>>>> it
>>>>> gets out.)
>>>>>> Hello all,
>>>>>> I'm afraid the news is not good. D2L reports that they have received
>>>> new
>>>>> devices at the hosted site, to fix the problem. There were to be
>>>> configuration changes expected to finish at noon which have not
>>>>> completed.
>>>>>> They expect it to be completed by 6pm ET, which seems to imply that
>>>>>> is
>>>>> when we will be back up. Our next update from D2L will be at 3:00pm.
>>>> Unless we hear anything more before then, that will be the time of our
>>> next update.
>>>>>> Andrea
>>>>>>> Hello everyone,
>>>>>>> We asked D2L to redirect our site to a page that says the system is
>>>>>> unavailable. We have not had an update since the message at 8:07am,
>>>>>> so
>>>>> still anticipate the system to be back by noon. We have asked that
>>>>> the
>>>> redirect page provide updated information, but cannot guarantee it.
>>>>>>> We will let you know as more information comes in.
>>>>>>> Andrea
>>>>>>>> Hello,
>>>>>>>> We have heard from Desire2Learn (8:07am) and have done some
>>>>>>>> testing
>>>> of
>>>>>>> our
>>>>>>>> own. According to D2L, they have "partially implemented" their
>>>>>> solution.
>>>>>>> They caution that some sites may still see impacts. They
>>>>>>> "anticipate
>>>>>> steady improvement to normal performance" by noon today.
>>>>>>>> From our tests, Waterloo LEARN seems to be very slow for login and
>>>> for
>>>>>>> accessing files. (At about 7am it was performing well, but is not
>>>> now.)
>>>>>>>> We will keep you apprised as we hear more. Our sincere apologies
>>>>>>>> for
>>>>>>> this
>>>>>>>> significant interruption. We will be pursuing detailed
>>>>>>>> explanations
>>>>>> from
>>>>>>> D2L.
>>>>>>>> Regards,
>>>>>>>> Andrea

More information about the Faccus mailing list