Barriers to CJ

Activity

1. Read the key points from the document below and reflect upon:
   • Which of these is the main challenge for you?
   • Which of these intrigue you / you would like to work on a solution for?
     (5 minutes)

2. Discuss with the person next to you and then with your table.
   Feel free to engage with people at other tables.
   (20 minutes)

3. Whole group discussion.
   (15 minutes)

4. Suggestion for follow-up for next meeting.
   (10 minutes)

Key Points and Themes from the responses:

1. Comfort and Enjoyment

• Some users find CJ to be a fun and engaging method, creating a comfort zone for them.

2. Challenges and Issues

• Finding Assessors: Difficulty in finding paying assessors.
• Mathematical Complexity: The maths involved can be challenging.
• App Integration: Lack of apps that integrate well into university VLEs (Virtual Learning Environments) and lack of configurability.
• Scepticism: Some reviewers and users are sceptical about the validity and transparency of CJ.

3. Advantages of CJ

• Simplicity and Efficiency: CJ is appreciated for its simplicity and efficiency.
• Explainability: It provides a clear and explainable method for capturing expert judgments.

4. Specific Issues

• Experimental Designs: Complex experimental designs can be fiddly with existing tools.
• Testing for Multi-dimensionality: Concerns about the best way to test for multi-dimensionality versus noisy measurement.
• Integration with Other Platforms: Difficulty integrating CJ platforms with tools like Qualtrics and JISC Online Surveys.

5. User Perspectives

• Humanities: Users in humanities find CJ intriguing for quantifying subjective perceptions and attitudes.
• Bias Concerns: Concerns about bias in the selection of objects for assessment.
• Time Constraints: Time is a significant barrier for many users, as CJ is often a side project.

6. Validation and Acceptance

• Validation: Validation of the approach is challenging, especially in educational contexts.
• Persuasion: Convincing others of the validity and usefulness of CJ can be difficult.

7. Practical Considerations

• Ethics and Anonymization: Ethical considerations and anonymisation of objects are important.
• Non-standard Objects: Issues with using CJ for non-standard objects like video or audio.
• Tool Configuration: Need for tools that can be precisely configured to control experiments.

8. Barriers to Wider Use

• Expert Judges: Requirement for expert judges can be a barrier.
• Financial Constraints: Financial constraints in the education sector limit the use of CJ.
• Academic Interest: Limited academic interest compared to industry interest.