What you're actually looking for is not a CMS, but an LMS, or "Learning management system".
This is a fairly well-established sector, quite separate from the CMS sector.
The most popular product I can recommend is Moodle. It does have a fairly high learning curve, but that will apply to any similar product as well, as the functionality required for creating and running online courses is considerably more complex than for running a simple CMS.
The other thing about products like Moodle is that they're typically not aimed at producing a page-driven website like a CMS would be. If you have a need for a good-looking site with CMS pages for your site as well as the learning content, then you may need to consider having a CMS as well as an LMS.
In this case, there are a number of LMS products that have been written as extensions/plugins/modules for the various popular CMS platforms. You may want to investigate some of these. Alternatively, you may want to use a well-established LMS product like Moodle for the course material on your site, and integrate it with a CMS for your front-end website. For this kind of scenario I have used Joomdle with good success in a recent project, which integrates Joomla CMS with Moodle LMS, providing single sign-on and various other integration features. However you will need to get yourself familiar with both of your main platforms before attempting to integrate them together.
Creating course material. This is the difficult part for any online learning provider - indeed, there are many companies that specialise in just this and do nothing else.
If you want to create fully interactive course material, you will need to look for tools that support the SCORM format. SCORM is the industry standard platform for online course material. It is basically a file structure and JavaScript API that is understood by all the major LMS platforms (including Moodle). There are many tools that will allow you to create your course material as SCORM packages, which can then be imported for use into any LMS platform of your choice.
The choice of tool for creating your SCORM packages is too broad for me to go into here; there are lots of them, depending on your budget, your needs, and your skill level.
It is also possible to create course material directly within your LMS platform. For example Moodle allows you to create multiple-choice quizes. Your course could be a case of hosting a video and then asking a bunch of questions about it. Or your course could require coursework to be submitted, in which case you would need to set up users as teachers within Moodle, with permissions to give marks to students.
I hope this goes some way toward helping you understand what you need to do. It is a very large topic though, and I've barely scratched the surface. But googling for 'LMS', 'SCORM' and related terms will give you a lot more to work on.