CLO Secrets Revealed

5 Learning Leaders Share Secrets to Transform 
Your Certification Program

Download PDFDownload PDF
illustration of two people at a table chatting

As a learning program leader, people look to you for answers.
 What keeps learners engaged with our course material? Which courses need new or updated content? How can we improve our exam prep and practice experiences to help learners feel more confident? Yet finding the answers to these questions isn’t always easy.

We talked to 5 Chief Learning Officers and VPs of Learning at the country’s most innovative associations and certification providers and asked for the secrets to their success. They shared perspectives and tips across 3 key areas:

3 key area of tips. 1. Leveraging data for certification program success, 2. Improving learner engagement (and outcomes) and 3. Collecting and acting on learner feedback

Each section below reveals a different secret you can use to push your programs forward, scale your business, and answer your toughest questions.

illustration of several people in a zoom call

Secret #1

Ask the right questions, and use data to find the answers.

Everyone loves to talk about best practices when it comes to data—how to collect it, where to store it, and even what to look for. But the reality is 73 percent of company data goes unused. It’s imperative that you have a partner who can help get you where you need to be.

Tools and technologies are designed to enhance decision-making for your learning programs. However, the process of collecting and interpreting information can often be overwhelming. The era of mere pass-or-fail assessments is long past. By focusing on the right questions, you can unlock transformative insights from your learning program data.

Where are your learners coming from? Where are learners struggling? How confident are candidates heading into test day? Most LMSs collect massive amounts of data, but this can create what Tom calls ‘the tyranny of choice.’



As one of the IT industry’s top trade associations and a trusted provider of 15 IT certifications, CompTIA was hyper-focused on two metrics. First, they wanted to improve learner competence and mastery within practice exams. Then, they wanted to identify and reduce the points of abandonment within their certification prep experience.



Tom and team have efficacy research in full swing and preliminary indications are that if someone scores well on their practice exam, there’s a positive correlation with them passing their exam and earning a great job in IT. They also wanted to analyze where learners would drop out of material so they could intervene and help get these learners back on track.

“We aren’t making data-driven decisions in a vacuum,” Tom said. “We collect data from our customers, instructors, students, and various subject matter experts in the industry, bring this data together, respecting all data privacy provisions, and work with our world-class research team to validate the data to help us make better product and business decisions.”

Tom Reilly from CompTIAperson holding phone with the CompTIA platform on it "BenchPrep’s partnership with Snowflake is transformational.
By unlocking access to our own data lake, we have more data
at our fingertips and most importantly, have access to that
information faster. The faster we can collect and analyze our
program data, the faster we can make product decisions. This
makes our learning products highly effective and ultimately we can make more students successful." Tom Reilly

Secret #2

Start small to create a strong foundation for your data strategy.

Whether your certification program is brand new or you’re looking for fresh strategies, you should avoid making assumptions about program data and learner engagement. Instead, start small with your data and figure out what you’re trying to understand.

Medical school candidates from 120 countries rely on the Association of American Medical Colleges (AAMC) for MCAT preparation and medical career guidance. To enhance prep products and resources, the AAMC continuously monitors learner data and engagement.

“When we first began our partnership with BenchPrep, we had an existing data strategy that we thought we’d just want to fit into our new program. But we probably should have started with a blank canvas,” Heather said. “We were trying to create too many reports to satisfy a curiosity without first understanding the questions we wanted to answer.”

Heather encourages learning leaders to have a goal in mind before diving in rather than “creating a narrative that you don’t know where it will go.” Defining short-term and long-term goals will help key in on the bigger picture questions.

“A lot of our initial product surveys were focused on a single product, so we had somewhere small to start,” Heather added. “But now, we include our entire product portfolio. Sometimes starting small scares people because they feel like they’re not doing enough—but we found it helped to create a stronger foundation for our feedback process and data strategy.”

Having instant access to program data allows the AAMC to make quick tweaks to their content strategy. At the end of each exam year, they also have a full picture of learner data and feedback to analyze and implement before the next exam year.

“Another thing we’re particularly interested in analyzing is barriers to completion,” Heather said. “We want to dig in on the students who don’t end up taking the test, and getting their feedback to figure out how we can make our products and programs more effective for these students.”

Heather Williams from AAMCillustration of a laptop"When it comes to your data strategy, figure out the big
questions you want to answer first. When starting out, we
tried to capture every piece of information and creating
multiple data visualizations without understanding what
we were even looking for. Have a goal in mind before trying
to capture data that you won’t be able to properly analyze otherwise.” Heather Williams

Secret #3

Ask the right questions, and use data to find the answers.

If you want to improve the value and effectiveness of your learning programs, you must collect student feedback and ensure they’re having a positive experience. But, as a leader, how can you make sure that you’re gathering the right type of feedback and making it actionable? When should you start collecting feedback, how should you analyze it, and how do you act on it?

Leslie De Leon has been an integral part of developing and managing the AAMC’s MCAT prep products, and one area she is fixated on is collecting and analyzing candidate feedback.“

From the get-go, even before we launched the MCAT Official Prep Hub, we were collecting feedback,” Leslie shared. “We leverage a variety of surveys, usability studies, and social media to gather feedback. Our learners’ voices contribute heavily to our product roadmap and have helped us to introduce new features and tools on the platform, improved navigation, and redesigned layouts of content.”

The AAMC leverages call to actions via banner messages within their Official Prep Hub, and they’re able to instantaneously analyze the results and patterns within their analytics. And students are really compelled to provide feedback—AAMC’s diverse feedback collection strategy has resulted in higher than average response rates.“

The AAMC also stays on the pulse of user feedback by leveraging social media to understand how students are studying and what resources are the most helpful. Keeping an open dialogue with end users ensures that learners are empowered to get the most out of the AAMC materials and be better prepared to start their medical school journey.

Leslie De Leon from AAMCAAMC platform on a laptop"One of the advantages of developing content for an online platform like this one is the ability to incorporate user feedback. When we see a pattern emerging through access to real-time platform data, we’re able to incorporate changes into the next available release. That way, we don’t have to wait until the following testing year to add a new feature or try something new—we’re empowered to make these data driven
enhancements on the fly.” Leslie De Leon

Secret #4

Candidates want an excellent
(not easy!) test prep experience.

Continuing education has become essential in virtually every industry and profession—more than 26 percent of employed adults in the United States hold a professional certification or license. Not only does certification lead to higher earnings, but it helps workers stay on top of best practices and trends. Certification and test prep is a critical component of continuing ed, but with so many considerations for building candidate experience, where do you start?

The CFA charterholder is one of the most coveted yet challenging certifications out there—more than 250,000 candidates spend more than 1,000 hours studying each year to pass their three exams. John Veitch’s focus is ensuring that CFA’s curriculum, mock exams, and custom study plans will help candidates be best prepared for exam day.

“Something interesting we’ve learned from candidate feedback is that they don’t actually want an easy experience,” John shared. “Candidates are proud to carry the certification, and while we want to create a positive learning experience, that doesn’t necessarily mean making it easier to pass or making it more likely that people will. We just want to be sure that the practice content best represents what they will encounter on test day.”

John recommends making content sophisticated as opposed to “making it opaque.” From the get-go, content needs to draw learners in, and John ensures candidates stay engaged by continuously evaluating and rolling out new features within CFA’s Learning Ecosystem. “We turn to discussion boards to help learners ask questions and form study groups offline and surveys to ensure we have a continuous feedback loop. And developing our mobile application has helped them to learn on-the-go,” John said.

At the end of the day, the CFA team wants to ensure candidates find the product helpful and that they’re achieving value from their exam preparation program to ultimately help them advance their own careers. And the program is helping to achieve just that: charterholders see an average of 15-20% salary increase as well as having a significant impact on career growth.

John Veitch from CFA Instituteimages of the BenchPrep platform"The BenchPrep platform is a huge win for our candidates: it allows us to build practice questions, mock tests, and other content our candidates can access across computers, tablets, and via a mobile application which enables us to reach candidates wherever they are. Plus, the rich program data helps us predict which candidates are going to pass and which need more structure and support. That allows us to create more positive and personalized learning experiences.” John Veitch

Secret #5

Celebrate success along the way.

Online learning can feel isolating. Courses and programs may take months or years to complete. Learners spend hours passing lessons and unlocking course achievements, and they can lose sight of the big picture. As a learning leader, how can you make sure your programs are motivating your learners to keep going? 89% of LMS users say they would be more engaged in eLearning if it was gamified. Fortunately, there are tools and engagement strategies to apply throughout the learning journey.

More than 43 million adults in the United States cannot read, write, or do math above a third-grade level. ProLiteracy is an international nonprofit organization that supports adult literacy programs through education, advocacy, and professional development. ProLiteracy roots their learning programs in real-life scenarios and keeps learners engaged throughout their journey.

“So much of our learning pedagogy follows scenario-based training. Working with adults in the basic education space, we’ve recognized that in order for our learners to grasp these concepts, we have to make it applicable to their day-to-day lives,” Sonja explained.

ProLiteracy’s flexible eLearning platform allows them to create engaging learning environments with games, badges, and more to ensure students can apply what they’ve learned to their workplace and career. “One thing we’ve learned from our community of learners is the importance of not only celebrating the big victories—like completing a course or passing an exam—but also the small wins. It helps to keep students motivated, and coming back to the platform to keep learning,” Sonja added.

Students are excited to download their certificates of completion to share on their social media, send to friends and family, and print out to frame in their homes to remind them of how much they’ve achieved. And ProLiteracy’s work is really helping to change lives and close the adult literacy gap—their learning programs help over 500,000 adults improve their literacy skills each year.

Sonja Chernos from ProLiteracyillustration of a desktop and a person interacting with it "We love leveraging BenchPrep’s gamification features for our community of adult learners. Things like games, badge systems, milestone points, certification of completion, learning reminders, and discussion boards really help to
engage our learners throughout their learning process. Recognizing the small achievements is a big motivator and helps to inspire our learners to keep going.” Sonja Chernos

Conclusion

No universal recipe exists for learning program success, and mistakes are inevitable, as our learning leaders demonstrate.

But with the right approach to data management, you can make faster decisions, build better products, and have a leg up from your competition. By implementing a continuous learner feedback loop, you’ll find new ways to surprise and delight your learners and keep them coming back for more. And with a solid content strategy in place, you can create unforgettable learning experiences.

Every learning leader has different expectations and program goals. Maybe you want to increase pass rates for your certification program, or expand your program to reach new audiences—your path to success may not look the same as your neighbor’s. But with flexible tools, technologies, and partners, you can achieve your own version of success just like these learning leaders have.

illustration of a person at a laptop

About BenchPrep

BenchPrep is an award-winning learning platform built for high-stakes certification training. Our learning platform is designed to help candidates feel more confident and prepared for their credentialing and certification tests bydelivering an intuitive, efficient, and engaging study experience.

BenchPrep helps associations drive revenue by making it easy for learners to search, discover, purchase, and enroll in online study courses rich with interactive learning experiences. BenchPrep’s data capabilities also help associations unlock valuable insights so they can make better, more data-driven decisions on factors that affect course content, candidate sentiment, and learner behavior.

Many of the world’s leading associations and credentialing bodies trust BenchPrep to power their online study programs, including ACT, AAMC, ASCM, CFA Institute, CompTIA, GMAC, ISACA, and many others. More than 10 million learners have used BenchPrep to attain academic and professional success.

illustration of a laptop

Take the eBook With You!

Download PDF

Check Out Some of Our Other Content

people icons

3 Stages of Creating a Continuous Learning Journey for Your Candidates' Long-Term Success

people icons

4 LMS Superpowers Every Association Needs