FutureUTM

Please note: The data for this study came directly from the six vendors reviewed. As a user, if you disagree with the presence or absence of features for a particular CMS product, please contact the vendor with your feedback.

Quick Jump To:

• Table 1. Comparisons of features among CMS products evaluated: Learning Tools.

• Table 2. Comparisons of features among CMS products evaluated: Collaboration Tools.

• Table 3. Comparisons of features among CMS products evaluated: Authoring Tools.

• Table 4. Comparisons of features among CMS products evaluated: Course Management Tools.

• Table 5. Comparisons of features among CMS products evaluated: Administration Tools/Costs.

• Table 6. Comparative scores on the presence and absence of specified feature sets.

• Corrections Submitted by Users and Vendors after October 13, 1999.

Blackboard

WBT Systems

VLE

MadDuck

WebCT, Inc.

Embanet

CourseInfo

TopClass v.3.1

VirtualU v.2.4+

WCB v.4

WebCT v.2

IntraLearn

Collaboration Tools

Discussion Options

Y

Asynchronous

Y

Email (one to one)

Browser/Forms

Built in

Browser/Forms

Browser

Built in

Y

List Servs/Newsgroups (many to many)

Y

N

N

Y(3p)

Y(3p)

Y

Text-based conferencing (many to many)

Y

Y

Y

Y

Y

Y

Bulletin Board (one to many)

Y

N

N

N

Y

Synchronous

Y

Chat

Y(3p)

N

Y

Built in

Y

Y

Archive

Y(3p)

N

N

Y

Y

Y

Whiteboard

Y

N

Y(3p)

Built in

Y

Y

Archive

N

N

Y

Y

Y

Teleconferencing

N

N

Y(3p)

N

Y(3p)

Y(3p)

Video

N

N

Y(3p)

N

Y(3p)

Y(3p)

Audio

N

N

Y(3p)

N

Y(3p)

Y(3p)

Live, text-based conferencing

Y

Y

Y

Y

Y

Y

File sharing

Y

Y

Y

Y

Y

Y

Email attachments

Y

N

N

Y

Y

Y

Message attachments

N

N

Y

Y

Y

Y

File storage

Y

Assignments only

Y

Y

Y

Y

Private directory on course server

Y

N

Y

Y

N

Y

Public file library

Y

Y

Y

Y

Y

Y

Work Group Areas

Y

N

N

Y

Y

Y

Group Web pages

N

N

N

N

Y

Y

Group conferences

Y

?

Y

Y

Y

Y

Team building

N

N

Y

Y(3p)

Y

Y

Blackboard

WBT Systems

VLE

MadDuck

WebCT, Inc.

Embanet

CourseInfo

TopClass v.3.1

VirtualU v.2.4+

WCB v.4

WebCT v.2

IntraLearn

Authoring Tools

Course planning

Y

Y

Y

Y

Y

Y

Course design

Y

Y

Y

Built in;3p

Y

Y

Course templates

Y

Y

Y

Y

Y

Y

Automated glossary

N

N

N

Y

Y

Automated course TOC/index

Y

N

Y

Y(3p)

Y

Y

Course search engine

Y

Y

N

Y(3p)

Y

Y

File management

Y

Y

Y

Y

Y

Instructor file storage

Y

N

Y

Y

Y

Y

Batch upload

N

Y

N

N

Y

N

Batch delete

N

Y

N

N

Y

N

Instructor guide

Y

Y

Y

Y

Y

Y

Course exemplars

Y

Y

Y

Y

Y

Y

Web search tools

N

N

Y(3p)

Y(3p)

N

Y

Multimedia capability

Y

Y

Y

Y

Y

Y

Blackboard

WBT Systems

VLE

MadDuck

WebCT, Inc.

Embanet

CourseInfo

TopClass v.3.1

VirtualU v.2.4+

WCB v.4

WebCT v.2

IntraLearn

Course Management

Instructor information pages

Y

Y

Y

Y

Y

Y

Course info/syllabus

Y

Y

Y

Y

Y

Y

Course calendar/schedule

Y

Y

Y

Y

Y

Y

Announcements/Bulletins

Y

Y

Y

Y

Y

Y

Student management

Student presentation/project pages

N

Y

Y

Y

Y

Y

Registration

Y

Y

Y

Y

Y

Y

Batch upload

Y

Y

Y

Y

Y

N

Batch delete

Y

Y

Y

Y

N

N

Attendance/Participation Tracking

Attendance

Y

Y

N

Y

Y

Y

Participation

Y

Y

N

N

N

Y

Gradebook

Student access to own data

Y

Y

Y

Y

Y

Y

Automatic assignment progress tracking

Y

N

N

Y

Y

Y

Assignment reminders

N

N

N

N

N

N

Automatic grade calculation

N

N

Y

Y

Y

Y

Class averages auto calc

N

N

Y

Y(3p)

Y

Y

Assessment

Quizes

Y

Y

Y(3p)

Y

Y

Y

Timed

Y

Y

Y(3p)

Y

Y

Y

Repeatable

Y

Y

Y(3p)

Y

Y

Y

Exercises

Y

N

Y(3p)

Y

Y

Y

Timed

N

Y(3p)

Y

N

Y

Repeatable

Y

Y(3p)

Y

Y

Y

Course Archive/Backup

Y

Y

Y

Y

N

Course replication

Y

Y

Y

Y

Y

N

Course revision

Y

Y

Y

Y

Y

Y

Online Help/FAQs

Y

Y

Y

Y

Y

Y

While we checked the accuracy of this report with representative from each vendor, it is always possible that misunderstanding of specific feature definitions may have resulted in answers that are later thought to be inaccurate by those same vendors. In addition, users often report that certain features a vendor thinks are present, either are not really there or don't work as the user would hope.

For both of the above reasons, FutureU invites vendor and user feedback to this report.

At FutureU we're always interested in a diversity of experience and opinion. If you want to join in a discussion about the pros and cons of the Course Management Software reviewed here, drop in to our Online Learning Exchange and join the discussion forum to post your contributions.

Please note that there was one error in the IntraLearn from Embanet chart...IntraLearn does Batch Loading (doesn't do Batch Deleting, however). The chart had it listed as a No

Steve
Director of Product Strategy
Blackboard Inc.

========= Eval Change ========================================== Area = Collaboration Tools
- Message Attachments
Courseinfo allows message attachments in both the email as well as the discussion board communication areas.

- Group Pages
CourseInfo allows users to create multiple groups and provides each group with their own set of communication tools as well as file sharing.

- Team Building
CourseInfo allows users to contact one another through the email specific users and allows them to collaborate in team environments through the use of group pages. Motivational team building is provided through group collaboration and file sharing.

Area = Authoring Tools
- Batch Upload
This technical facility allows users to batch upload a group of files that may exist in a web structure or a multitude of files that are interconnected.

Area = Course Management Tools
- Student Presentation/Project Pages
The group areas can be used for either groups of students or single individuals and allow individual students to post documents and other project files that they have created during the course.

Automatic Grade Calculation
- CourseInfo Automatically calculates the students grade as they take tests and quizzes in the system. In addition this infromation is stored in the online gradebook.

- Timed Assessments
Instructors to allocate timed sessions for tests and quizzes.

Area = Administration Tools
- Automated Registration
CourseInfo provides all the necesssary tools required to batch upload students to the system.

- Claude you probably can't see this at a system basis as you need administrative access. I would be more than happ y to provide you with temporary access if needed, or I could show you directly at EDUCAUSE.

- Student Transcript
A transcript of how the student is doing in the course is stored in the gradebook and can be printed upon request. Simply search by user and pull up all their grades for the course.

- Demo/Real Course
Instructors can create a course on Blackboard.com for no cost and can keep the course their indefinitly.

Dear Ms. Lozano:

Thanks for voicing your concern. At FutureU, we appreciate and encourage a diversity of opinion.

FutureU is an independent consulting firm and content provider whose e-books and courses run in any Course Management System.

We feel very fortunate that MadDuck technologies was generous enough to host us in their booth at Educause. But as I am about to explain, our relationship with MadDuck had nothing to do with how well they did in our comparison report.

If you will read the "Methodology" section closely, you will find that we sent the features list to each vendor and the answers we include in the report are the ones that the vendor reported. We asked all vendors to report on their most current versions.

The answers included for WebCT came from Phillip Chatterton, Sales & Marketing Coordinator, ULT Canada (WebCT's parent company).

There are two logical ways that you can get your opinions about WebCT heard:

1) Contact WebCT and let them know that you disagree with the answers that their agent supplied to us. Urge them to send a new set of answers. When we receive those new answers, we will update the report to reflect them (after some testing to be sure that they actually work).

2) Join a discussion forum in the FutureU Online Learning Exchange to let your opinions be known to the online educators participating there. We will open the Learning Exchange discussion forums sometime this month and will be happy to send you an invitation when that happens. In the meantime, you can get a sneak peak at the whole virtual community for online educators at http://www.futureu.com/olx.html

Thanks again for your concern. We welcome your ideas on any subject related to online education and we look forward to hearing more from you in the near future. Your interest and attention to detail would be a valuable addition to our Online Learning Exchange. I very much hope you will visit the site often.

November 17,1999

Claude Whitmyer, CIO
FutureU.com

Dear Claude -

We recently became aware of your comprehensive report comparing course management systems -- it's a terrific feature by feature comparison - we'd just like to clarify a few points and make some corrections to the data regarding WebCT; the corrections are based on the current v.1.3.1, but apply also to the new version 2.0, which is being released within the week.

First, you may not be aware that, following the merger of ULT and WebCT, the corporate name became WebCT, so that is one change we'd like to request on your site, if possible.

I will run through the feature discussion in the order they appear in your report -- the cumulative change,in terms of WebCT's score is to place it at 86% of the total possible. We do feel this is a conservative score, given the over 100 feature improvements that are found in v.2.0 -- as is always the case, the limited descriptions of the features make it difficult for such a comparison to truly reflect the differences between the products. We appreciate the fact that your site encourages further discussion and information sharing.

Learning Tools
Student area
Private Directory on course server: Should be "Y". The Student Presentation area can be set up for individuals or groups, and students may use it for individual file storage if they wish.

File Storage: Within Student Presentation area.

Private Directory on course server: unclear as to meaning. Every course has its own discrete site, with a wholly functional file management system unique to the course.

Total for Collaboration Tools: 15 (leaving out Teleconferencing,audio, video, file storage, and private directory, because of unclear definitions).

Authoring Tools:

Automated course TOC/index: Should be "Y". This feature is further enhanced with the dynamic site map included in v.2.0.

Batch upload: Should be "Y". Built in zip utilities allow easy upload of course files.

Total for Authoring Tools: 12

Course Management Tools:

Announcements & Bulletins: Should be "Y". Instructors have had many ways to make a announcements; with v.2.0, announcements and bulletins can be sent from either the instructor or the administrator level.

Batch upload: Should be "Y".

Participation: Should be "Y". The system reports how many messages each student has accessed on the Bulletin Board, as well as how many postings made.

Assignment reminders: The Calendar allows assignment information to be posted in an "at a glance" format.

Quizzes: Timed: Should be "Y". WebCT's quiz tool has always had the option of setting up timed quizzes.

Course Archive/Backup: Should be "Y". Courses can be backed up and archived by the instructor or the administrator. Built in utilities make this possible in a matter of minutes.

Course Replication: Should be "Y". Any course on a WebCT server can be a template for a new course. There is the possibility for infinitely replicating any WebCT course.

Total for Course Management: 23

Administration Tools:

Automated Registration: Should be "Y". WebCT currently allows for easy importing and exporting of student data. With v.2.0, an open API makes this process even more transparent; additionally, WebCT has partnerships with SCT/Banner, and PeopleSoft, for development of specialized APIs for their products.

Student Transcript: Should be "Y". Students instructors and administrators are able to access and download course records. The information can be printed or imported into another database.

Standard: Open and Proprietary.

Hosting Services: Should be "Y".

Demo course: "Y, free".

Platform: Windows NT: Should be "Y".

Total Administrative Tools: 15

The annualized license fee is described in detail at our website. Most importantly, the maximum cost is $3,000 for unlimited number of students. Please reflect this change in the table.

With recalculation, WebCT's total is 81 and the total percentage of attained features is 86%. These totals do not include credit for any features where the definition was in question, as identified in the notes above.

Thank you for your time with this, Claude -- hope this clarifies some of the questionable areas.

Do let me know if there's additional information I can provide -

Regards,

Kathleen, Thanks for your clarification letter. We will integrate your answers into the report when we next update it, which should be within a week to 10 days.

claude

P.S. FYI: As you may have noticed from the Methodology section, we submitted a table of features to all vendors and used the vendor answers to create our comparison. You may want to contact Phillip Chatterton, Sales & Marketing Coordinator, ULT Canada, and inform him of the differences you have reported to us so that other researchers will receive a uniform answer from within WebCT in the future. (Our email was routed to him from "support@webct.com" in case you want to also alert them about who should have actually rec'd our request).

Message : Your comparison report was extremely unprofessional. You made statements concerning features that WebCT does or does not have which were incorrect. WebCT does have a Bulletin Board, student registration upload, e-mail attachments, and student participation records. You failed to mention that WebCT does automatic grading with a built in grade book. You also made the statement that "students prefer" a particular type of mail structure without any explanation on how you determined this. This was a very irresponsible survey that was published. I can only pick out the erros that were made wih the package that I am presently using, but I imagine that you have made many mistakes in the other packages that you did not want to use.

Dear Ms. Alexander:

Thanks for voicing your concern. At FutureU, we appreciate and encourage a diversity of opinion.

FutureU is an independent consulting firm and content provider whose e-books and courses run in any Course Management System.

If you will read the "Methodology" section closely, you will find that we sent the features list to each vendor and the answers we include in the report are the ones that the vendor reported. We asked all vendors to report on their most current versions.

The answers included for WebCT came originally from Phillip Chatterton, Sales & Marketing Coordinator, ULT Canada (WebCT's parent company). According to officials at WebCT these answers were grossly inaccurate.

A second set of answers was sent to us in mid November by Kathleen Paul in WebCTs Boston office. These have been added to the table page located at: http://www.futureu.com/cmscomp/cmstables.html#corrections.

There are two logical ways that you can get your opinions about WebCT heard:

1) Contact WebCT and let them know that you disagree with the answers that their agents supplied to us. Urge them to send a new set of answers. When we receive those new answers, we will update the report to reflect them.

2) Join a discussion forum in the FutureU Online Learning Exchange to let your opinions be known to the online educators participating there. We will open the Learning Exchange discussion forums sometime in January and will be happy to send you an invitation when that happens. In the meantime, you can get a sneak peak at the whole virtual community for online educators at http://www.futureu.com/olx.html

Thanks again for your concern. We welcome your ideas on any subject related to online education and we look forward to hearing more from you in the near future. Your interest and attention to detail would be a valuable addition to our Online Learning Exchange. I very much hope you will visit the site often.

Dear Sir/Madam. I am Murray Goldberg, founder of WebCT and president of WebCT Canada. I read with great interest the comparative evaluation you have posted at http://www.futureu.com/cmscomp/

These evaluations can be very useful and as such are relied upon by people entertaining the idea of adopting such technologies. Unfortunately, the information in your review of WebCT is horribly and grossly inaccurate. I am not referring to a few items where judgment calls may lead the presenter to one conclusion or another. I am referring to a very large number of clear and significant errors. I question whether it was actually WebCT being reviewed. One of many clear examples of error is whether WebCT has a bulletin board - the comparison says no - but WebCT was *built* on its ability to communicate and no user of WebCT could possibly ever report that it had none. Kathleen covered most (though not all) of the largest errors in the report, so I will refrain from restating them here.

Could I please ask that you:

1) let me know which employee of WebCT provided you with these incorrect answers
2) either update the site immediately with correct information, or if that is not possible, remove WebCT from the listings. No information is better than grossly inaccurate information.

In order to obtain correct information, please feel free to call either of the following people:

Myself - Murray Goldberg. I will be available this week in either the Vancouver or Boston office.

Sasan Salari - VP WebCT Canada - reachable at the same numbers as above

Alternatively, we could provide a written response if you preferred that. This would be my preference. Let me know - either way I would very much like to see you with correct information immediately. In the interim, Kathleen's response is accurate and can be used. Only small errors would remain at that point.

Let me apologize if the tone of this letter is abrupt, I was simply shocked that you could have been provided with such inaccurate information. I look forward to hearing back from you and helping to rectify the errors in the report.

Best regards - Murray

12/15/1999
Dear Dr. Goldberg:

I just tried calling your Vancouver office and they informed me that you were on your way to Boston. So I'll give you a call there tomorrow.

I want to immediately reassure you, however, that we will do all we can to make sure that the facts about Web CT are accurate.

If you read the methodology section of our report, you know that we sent the features list to each vendor and the answers we included in the report are the ones that the vendor reported. We asked all vendors to report on their most current versions.

The answers included for WebCT came originally from Phillip Chatterton, Sales & Marketing Coordinator, ULT Canada.

A second set of answers was sent to us in mid November by Kathleen Paul in WebCT's Boston office. These were added right away to the table page located at: http://www.futureu.com/cmscomp/cmstables.html#corrections.

We hope to integrate these answers into the text and tables themselves before the end of the year, so it would be greatly helpful if you could provide your next set of written corrections as soon as possible.

Since your email arrived an hour ago, I have also added a pointer in the Revisions table on the main report page so that it is more obvious how to find Kathleen's data. Please also note that extensive corrections were provided by one of your users, Professor Joseph Delaney at the Teaching Excellence Center of Rutgers University, back in mid-October and these were also added to the corrections as soon as we received them. Also, Blackboard submitted a second set of answers in October which we have already integrated into the narrative and tables.

I present this information as evidence of our good intentions. From the beginning, we designed the report to be as fair to all vendors as possible. We are most interested in an honest and open exchange between users and service and product providers. Our hope is to stimulate excellence in all the products available.

FutureU is an independent consulting firm and content provider whose e-books and courses run in any Course Management System. In January we intend to open a virtual community about teaching and learning online entitled the "Online Learning Exchange" with discussion forums and resource pages specifically focused on the needs of administrators and instructors specializing in online learning. We are sending invitations to all major CMS vendors to invite their own customers to visit the Learning Exchange so that actual users can speak for the software they have chosen. We expect this to be quite useful to both vendors and customers, by providing detailed feedback about the pros and cons of each CMS product.

You can get a sneak peak at this virtual community for online educators at http://www.futureu.com/olx.html

I look forward to speaking with you more about all of this tomorrow when I call.

All the best,

claude

12/16/1999
Dear Dr. Goldberg:

I called again today and will await your return call and/or email with written corrections.

I look forward to receiving your new feature set description and will hold off editing the comparison report any further until I have heard from you.

Sincerely,

claude

12/27/99
Dear Dr. Goldberg:

I will be rewriting the CMS comparison report in the next few days. If you are able to provide me with your latest feature set information, I would be happy to include it. Otherwise, I will use the information sent to me by Kathleen. I will need what you want to provide no later than this Thursday (12/30) as I intend to finish this project over the New Year weekend.

By the way, the California Virtual Campus has asked us to make a presentation this spring on how to choose course management software. We have gotten them to agree to invite representatives from WebCT, Blackboard, Web Course in a Box, TopClass, and Intralearn to make mini-presentations to our audience. Were not sure of the format or time allotted for each vendor, but you should receive an invitation very soon with the details.

I will look forward to meeting your representative or you, if you happen to attend.

Sincerely,

claude

At 03:05 PM 12/29/1999 -0800, you wrote:
Dear Claude,

Murray Goldberg is out of the office, so I am responding in his stead. I re-read Kathleen's comments, and they are accurate descriptions of WebCT's features. I thank you for making the changes in your report and look forward to the update.

Best regards and a Happy New Year,

Sasan

12/30/99

Dear Mr./Ms. Salari (I apologize for not recognizing the gender of your name, but my language abilities are limited mostly to English, I'm sorry to say):

Dr. Goldberg seemed clear in his original email that he wanted to expand beyond Kathleen's answers. But he also said you were the person to talk to about this if he was not available. So, based on your reply, I am going to go ahead with Kathleen's comments.

The only drawback to this is that if Dr. Goldberg wants additional changes they will have to wait until the next revision of the report, which will probably be the annual update scheduled for next fall. He might try emailing me his comments over the weekend, but I can't guarantee I'll see them if they come in after Thursday.

Sincerely,

claude

Date: Thu, 30 Dec 1999 11:54:10 -0800
From: Sasan Salari
Organization: WebCT, Inc.
To: Claude Whitmyer
CC: "Grimes, Gail Terry"
Subject: Re: [Fwd: Request for Changes]

Dear Claude,

it's Mr :) (but you can just call me Sasan - no need for formality).

I would be grateful if you could use Kathleen's comments as a basis for the update. I have made a couple of additions/modifications to Kathleen's answers below, so my comments are in addition (not a substitution) to Kathleen's feedback.

Version: we just released version 2.0, so that would be the best one to use in this comparison.

Collaboration Tools:

List Servs/Newsgroups: while WebCT does not itself provide a listserv or newsgroup servers, but instructors can easily link to existing ones from their course web-pages.

Public File Library: the student presentation area can be used as a way for students to distribute files either publicly to their classmates or privately to their own group members, so I would classify this as a "Y".

Batch Delete: should be a "Y" as students can delete more than 1 file at a time.

Authoring Tools:

Batch Delete: should be a "Y" as instructors can delete more than 1 file at a time.

Pricing:

The price range for a WebCT server annually is $100 to $3000, depending on the total number of student seats,where $3000 provides an institution with an unlimited license. In your pricing example for 100 students annually, the cost would come out to $500 per year.

Have a great weekend and thanks for all your efforts,

Sasan