Evaluation Revision Notes

The first step in evaluating an ICT solution is to check how well it meets the original task requirements. These requirements define what the system was intended to do, including specific functions, performance targets, and constraints.

For example, if a school database system was required to store student records, generate reports, and allow secure access, the evaluation should verify that all these functions are present and work correctly.

This comparison ensures that the solution delivers the expected outcomes and helps identify any missing features or incomplete tasks.

• Review the original specification or brief.
• List the required functions and performance criteria.
• Test the solution to confirm each requirement is met.
• Note any missing or partially implemented features.

Effectiveness refers to how well the solution performs its intended functions. It involves checking accuracy, speed, reliability, and completeness.

For instance, if the solution is a data entry form, effectiveness includes whether it accepts valid data, rejects invalid entries, and processes information correctly.

Testing should include realistic scenarios to observe if the system behaves as expected under normal and peak conditions. Any errors, crashes, or incorrect outputs reduce effectiveness and must be recorded.

• Test all functions with valid and invalid inputs.
• Measure response times and processing speed.
• Check for errors, bugs, or data loss.
• Confirm outputs are accurate and complete.

User-friendliness assesses how easy and intuitive the system is for its intended users. This includes the interface design, navigation, instructions, and error messages.

For example, a mobile app designed for elderly users should have large buttons, clear labels, and simple workflows.

Observing users during testing or collecting their feedback helps identify confusing elements or obstacles. A system that is difficult to use may cause frustration and reduce productivity, even if it is functionally effective.

• Examine the interface layout and design.
• Check clarity of instructions and labels.
• Observe users performing typical tasks.
• Collect feedback on ease of use and satisfaction.

Suitability considers whether the solution fits the context in which it will be used, including hardware compatibility, software environment, and user needs.

For example, a solution designed for a school should be compatible with the available computers and network infrastructure, and meet the skill level of staff.

If a system requires high-end hardware but the organisation only has basic equipment, it is unsuitable despite good functionality. Similarly, a solution that is too complex for users is unsuitable for that environment.

• Check hardware and software compatibility.
• Assess if the system matches user skills and needs.
• Consider environmental factors like network speed or security requirements.
• Identify any mismatches that limit practical use.

After assessing effectiveness, user-friendliness, and suitability, identify any weaknesses or limitations.

For example, slow response times, confusing menus, missing features, or compatibility issues.

Suggest practical improvements such as simplifying the interface, adding missing functions, improving error handling, or optimizing performance to better meet requirements and user expectations.

• List all problems found during evaluation.
• Prioritise issues based on impact and feasibility.
• Propose specific changes or enhancements.
• Consider cost, time, and resources needed for improvements.

User feedback collected during testing provides valuable insights into real-world use and user satisfaction.

Feedback can be gathered through surveys, interviews, observation, or usability testing sessions.

Analyse comments and ratings to identify common issues or suggestions for improvement. This helps validate the evaluator's findings and guides further development to better align the solution with user needs.

• Collect feedback systematically from representative users.
• Analyse feedback for recurring themes or problems.
• Compare feedback with evaluation results.
• Use feedback to refine and improve the system.