Project Overview
Many international students, like domestic students, come to NYU for the excitement of living in the greatest city in the world, for top-notch academics, and for the promise of bright future.
NYU, after all, is renowned for its career and internship opportunities. The university is in the heart of New York, where virtually every industry is a subway ride away. Its award-winning Wasserman Career Center is known for its services, alumni connections, and top-notch career outcomes.
Yet, international students face more barriers to these resources and outcomes than domestic students. Of the 86% of graduate students who report working or continuing their education after graduation, only 41.1% were international graduates, compared to 58.5% who were domestic. And only 79.4% of international graduates remain in the United States, compared with 98.5% of domestic students. [Source]
International students are a diverse and important part of NYU's student population, and NYU needs to deliver top-notch career outcomes for them, too. While career services exist for students, there is a lack of customized learning solutions that address the greatest barriers international students face. Therefore, a learning solution is needed specifically for international students who are looking to secure employment in the United States.
This is our key question and challenge: how can we help international students at NYU secure a job in United States?
Design Process(ADDIE model)
ADDIE consists of 5 phases:
-
Analyze. In this stage, we conducted our learner and content analysis, which we used to refine our initial learning goals and objectives.
-
Design. In this stage, we set the instructional design, activities, media and technology approach to our design, grounded in relevant learning theories. We followed the the double-diamond to diverge and converge on ideas, ultimately narrowing down from our initial design ideas to our final design solution and prototypes.
-
Develop. Due to the time constraints of the project, we focused on developing only one part of our learning solution more fully: our professional communications module. We focused on developing prototypes of our elearning course on building professional communications with an AI tool for feedback and practice.
-
Implementation. Due to the time and academic constraints of the project, our implementation stage was limited to preparing materials for and training for test learners.
-
Evaluation. In this phase, we developed an evaluation plan. Due to the time constraints of the project, again, it focused on only one module and 2 methods of observational user testing and interview. We conducted user testing with 6 individuals. We recorded user testing sessions and compiled task completions, utterances, and analysis in observation reports. We conducted post-testing interviews with all users, compiling verbatims into a Miro board and using affinity mapping to identify key themes to action on in subsequent revisions.
Empathy was key throughout each of these five stages, from conducting our learner analysis to affinity mapping in our evaluation plan.
Analysis
Learning goals and objectives
Goal 1: Secure a job in the U.S.
-
Identify sources for job openings
-
Organize job info
-
Adhere to job and visa application timelines
-
Build an effective resume
-
Write an effective cover letter
-
Deliver effective supplemental materials, as needed (e.g. writing sample, portfolio)
-
Submit online job applications
-
Improve interview skills
-
Develop networking skills
-
Improve business communication skills
Goal 2: Apply for work authorization correctly
-
Differentiate different types of visa (CPT/PRE-OPT/POST-OPT/H1B), and the timeline for each one
-
Identify the correct visa status for your situation
-
Communicate with advisors
-
Fill out paperwork from the international center
-
Recognize the steps to apply for correct visa
Learner Analysis
We conducted a survey of 36 international students at NYU spanning undergraduate and graduate populations.
In addition, we conducted interviews with 2 international students who are looking for employment in the U.S. after graduation, 2 former international students who have already secured employment in the U.S.
We also spoke with 2 subject matter experts at NYU.
We also gained valuable voice of the learner anecdotals that validate and build upon the themes that emerged in our survey results and the gaps we identified in our content analysis.
"Networking is very important." - Former student
"Improve your English proficiency." - Former student
"I don't know how to reach out to people; it's awkward." - Current student
"I don't know how to build a U.S.-style resume." - Current student
"The career center doesn't really help. They just gave me templates." - Current student
In addition, across all interviewees, visas and work sponsorship came up as a common pain point. Current students we inteviewed, like survey respondents, shared that that are simply not confident in the job and visa process, that they face challenges with language barriers, and are intimidated by interviews, since they have not experienced them personally.
Content Analysis
Our team performed a content analysis on information and learning resources available to international students seeking employment in the U.S. after graduation. Our analysis is broken out into two topics: analysis of career preparation content for international students, which includes sub-topics like interviewing, network, resume building, and more; and analysis of visa and work authorization resources.
Design &
Development
Early Design Idea
Design #01: Resume building module
Design rationale
-
Resume-building is a critical step along the way to securing employment in the United States, and two of the top challenges cited by our target learners directly affect resume building: lack of work experience and language barriers. As one learner said, "I don't know how to build a U.S.-style resume." From this module of our learning design, learners will build an effective resume.
Flexibility and accessibility (technology choice and rationale)
-
Our solution encompasses both asynchronous and synchronous learning. It includes both an an e-learning platform and an in-person workshop with peer and expert feedback. Students will use document editng software of their choice (on laptops) to work on their resumtes.
Mentoring and feedback
-
Throughout the e-learning, students will get feedback on assessment questions they answer correctly and incorrectly and engage with examples that will be contextualized with callouts on best practices.
Learning theories
This module is based on Constructionism, Cognitivism.
-
Learners would be able to create their own resumes.
-
Instruction is pre-designed, and learners will be actively involved in recalling previous knowledge and mapping new knowledge in a meaningful way.
Outline of learning experience
Through a gamified e-learning experience, learners can discover the key components of resume, evaluate examples, and apply best practices to an early draft they build on their own resume prototype within the platform. The rationale behind this experience is that learners should have the flexibility to do this on their own time. Through this activity, learners will be able to
-
Articulate different types of resumes used in the US and their purposes
-
Analyze job postings and role/employer's requirements
-
Identify transferable skills, even without significant work experience, that they can feature and tailor for their resumes
To ensure its relevance, we will release the module 2 weeks prior to the in-person workshop and require that students complete it prior to attending, so that learners are fresh and prepared for the in-person component featuring peer and expert feedback.
During the workshop, an expert presenter will lead learners through an interactive lesson on resume best practices. Learners will pair off to workshop their resumes further through peer review. Then career coaches will provide 1:few expert feedback on resumes. Through this activity, learners will:
-
Contextualize best practices to their personal resumes
-
Build an effective resume
[Storyboard of resume building learning design]
Final Design Solution
After careful consideration and thorough discussion, we have decided to focus on professional communication and divide our design into two parts. The first part is an E-learning course that covers four models that are closely related to the communication of the job search process for international students. In the second part, we incorporated cutting-edge AI technology to not only evaluate students' learning outcomes from taking the course but also help them to practice by providing constructive feedback.
Why the e-course + AI ?
In our research phase, we captured the gap between targeted learner and resources that helps to achieve their ultimate goal: cultural gap, language gap, and stress. It is not because there are not enough resources and faculties to support them. Instead, there are plenty of resources from academic advisors, the OGS office, and the Wasserman career center. However, due to the lack of knowledge of how the process is in job-hunting, learners do not have the initiative to reach out for those resources. For instance, learners do not know they need to write a cover letter besides of resume; they do not understand the importance of networking in the U.S., and how to make professional connections.
With the lack of prior knowledge, it is hard for them to actively reach out to related resources. Thus, we decided to create an online course with Rise 360, consisting of 4 modules (Email and cover letter writing, follow-up conversation, networking, and job interview) to hone their written and verbal communication skills. After taking the course, we also designed an AI practice platform and mock interview sessions with mentors to practice those skills.
The course is designed to be embedded into NYU Brightspapce, the LMS system NYU is using. In this case, international students will have access to the course with the least effort.
E-Learning Course
In the e-learning course, we designed 4 modules to hone international students' written and verbal English communication skills. Those 4 modules include email and cover letter writing, follow-up conversation, networking, and job interviews. In our early development phase, we first came up with the course plan, and then created storyboards for video lectures based on the plan.
Course Plan
Prototype on Articulate 360
Feel free to explore more about the course in RISE 360 version!
AI Practice Tool
This is a display of cold message practice, generated with the AI practice tool.
The AI practice tool is set to be "resistant", and demonstrates push back when our learner first reached out.
Evaluation
Audience:
The audience of this evaluation plan will be senior-level stakeholders and funders of NYU career services. They include:
-
Linda G. Mills, Vice Chancellor for Global Programs & University Life
-
Gracy Sarkissian, Executive Director, Wasserman Center
-
Deans and Executive Directors of individual NYU school career and professional development services
-
Program funders
Questions to be answered:
-
Usability: What's working well in our design? Are learners taking the appropriate amount of time to complete activities? Where are learners getting stuck?
-
Conceptualization: Should the design achieve our stated learning goal of improving business communications and outcomes of increasing interview offers?
-
Learner effectiveness: What knowledge, skills, attitudes did learners learn? Has knowledge transferred? Have learners applied what they have learned?
-
Program effectiveness: Are more students participating in our training? Did career outcomes for international students improve?
Methodology:
We will take several approaches to measure usability:
-
We will conduct an observational usability study with test learners as a formative evaluation on usability. We will record our study to refer back to, to identify patterns in behavior.
-
We will conduct a semi-structured interview by asking post-testing questions as a formative means to understand learners' reactions to the tool. Question examples include "What was the experience like interacting with the AI?" and "Did you feel like the effort to craft the messages you were prompted to write was too much or too little or just right?"
We will use several approaches to evaluate learning outcomes:
-
Our learning design will incorporate quizzes throughout and results summaries on the AI practice portion for formative evaluation.
-
Mentors will provide feedback to students after mock interview sessions and in networking events.
-
We will use Kirkpatrick's Training Model to evaluate learning outcomes in students' reactions, learning, behaviors, and goals.
We will enter usability results and feedback into a shared Miro board and code our data for themes. We will track and analyze qualitative results in shared Google sheets.
Justification:
This evaluation plan is both formative and summative. We will conduct user observation to ensure the product provides a smooth and easy interaction for users. While assessing learning outcomes, questions are based on Kirkpatrick's Training Model that evaluates four levels: reaction, learning, behaivor, and results. By answering these questions, we will form improvements to our learning design and content, and measure program effectiveness and impact.
Reporting:
We will launch in beta to 60 initial users and use this period. We will use Asana to report on bugs and any results below the benchmark, and use an agile approach to implementing improvements to our learning design. We will launch for general availability following this period of initial testing. After receiving our 6-month survey results, we will submit a formal report on our findings to our VPs and funders.