Assessment Guide - Capstone Project

Introduction

In the Capstone Design Project, you will practice an authentic project evaluation process.

Timelines

Capstone takes place over two semesters. An initial semester, the semester in which you commence your project and a concluding semester when you complete your project. The timeline for assessments is as follows:

These timelines may be adjusted if ANU and health restrictions change. Where possible events will delivered in-person with a real-time online delivery via Zoom available only if required.

Project Audits (PA)

Project audits are used to evaluate the progress of each team

Project audits are a standard practice in many industries for evaluating project progress and ensuring alignment with objectives. In the Capstone Design Project, each team’s project is unique—some may involve design and build tasks that advance through multiple stages of the engineering systems design lifecycle, while others may be more conceptual, focusing on the early stages of the lifecycle. Teams also adopt varied approaches to managing their projects, reflecting the nature of the work, the composition of their team, and the needs of their stakeholders. The project audit process accommodates this diversity, providing a structured yet flexible framework for evaluation. It ensures all projects and teams, regardless of their differing outputs and processes, are assessed fairly. Additionally, audits offer valuable formative feedback to help teams meet their project goals while maintaining high standards in systems engineering and project governance.

The goal of the project audits is to enhance your projects through constructive, actionable feedback with critical and considered review. The Systems Engineering Body of Knowledge (SEBoK) (Chapter 10, 2.3) provides a good overview of why Project Audits and Reviews improve the technical aspects of projects.

The Project Audit contains three elements:

1. Project Repository (the evidence) – work product and activities, including the work done and governance of your project,
2. Project Review (individual audit submission) – a self-reflection and critique of your shadow team’s work against two criteria,
3. Team member contribution (individual audit submission) – an evaluation of the relative contributions of your team.

During tutorials in an audit week, you will provide a short presentation on the current state of your project work and governance. The audit review is not of this presentation, but the evidence of work done demonstrated within your repository against the specified criteria.

The Project Audit processes result in the award of progress indicators (Note: these are not grades), which may be used as evidence towards your final project grade.

The audit cycle

Project audits take place four times over the project in weeks 4 and 11 of each semester. Each audit week will run through the same cycle.

Key audit items

Project Repository

The purpose of the repository is to store all your project documentation and governance in one location in an organised and easy to navigate format.

These are the minimum requirements; each repository may contain additional information as needed. Teams are free to decide tools and storage but should:

• Consult with your project host – regarding preferred tools, access management, and costs.
• Access – ensure the repository is accessible to shadows, tutor, and convener. Consider confidentiality.
• Research templates – for meeting minutes and other documents to adapt for use.
• Transitional arrangements – ensure handover for next teams or host with access and control over tools.

Many groups opt to use the ANU’s license of OneDrive as the default repository system.

Landing Page

The landing page collates information about your project in a clear, accessible format. It must be submitted via Wattle and openly accessible to the teaching team and classmates. At minimum, include:

• Project summary
• Team members list
• Link to your repository (with access instructions if needed)
• Link to your Concept of Operations
• No actual project work (kept in repository)

Update before each audit cycle (or more frequently). It can also keep stakeholders informed through updates or blog posts. Tools like Wix, WordPress, or Notion are encouraged. Avoid confidential information due to public nature.

Concept of Operations

The ConOps is central to Project Audit 1 and defines the project goals. It outlines the what, why, and how of the project—not a technical design but a shared understanding of objectives. It should be developed with input from the project host and tutors. For Capstone, the ConOps serves as the official definition of your project.

Develop the ConOps as a mostly self-contained reference, with links to detailed repository sections if necessary. Obtain sign-off from the project host, tutor, and all team members—like a contract with signatures and dates. As the project evolves, update the ConOps with agreement and re-sign-off from all stakeholders.

Expectations/Requirements for each audit

The following sections are a guidance teams can use to benchmark expectations at each audit.

Project Audit 1 (PA1)

In addition to the initial creation of your repository and landing page, audit 1 should include at least the following essential information navigable from your landing page or repository:

• all work done, accessible and navigable within our project repository,
• the initial version of the ConOps
• templates for project governance (e.g., meeting minutes, decision logs etc.)
• an initial project management plan including:
  • Project scope and deliverables
  • Assumptions and constraints
  • Stakeholder analysis
  • a project schedule or timeline indicating dependencies (weekly table or gantt chart)
  • a work breakdown structure for the milestones and deliverables (as outlined in the ConOps)
  • a risk management plan. This should include project, technical and other risks (for example, reliability, security, safety)
  • Resources and budget (if appropriate)
  • Team structure
  • Communication Plan

Project Audit 2 (PA2)

Audit 2 is midway through your project. You should be progressing the project and providing value to your project host. Your review will be explicitly against the expectations set out in your ConOps. This audit is scheduled near the Mid-project Presentation; however, they have different purposes and a different audience. The purpose of a presentation during the audit is to highlight your recent progress and project governance to your shadows and tutors. The mid-project Presentation is to a wider audience and its purpose is the demonstrate the value and purpose of your project and your planned solution.

Audit 2 should include information navigable from your landing page or within your project repository:

• up-to-date landing page
• all work done, accessible and navigable within your repository
• baseline systems engineering governance: a requirements analysis, functional analysis, and/or system architecture, and/or modelling as appropriate for your project.
• technical artefacts: initial prototypes or models, or drafts of project outputs.
• progress against criterion specfied in the ConOps.
• any updates to the project management plan
• plan for achieving value in the remainder of the project.
• updates to project and WHS risk assessments if required.

Any updates to your ConOps should be undertaken with approval for your project host, tutor and team members between audits.

Project Audit 3 (PA3)

Audit 3 is past the mid-point of your project. Your review will explicitly be against the achievement of expectations set out in your ConOps (and any approved changes). Audit 3 should include information navigable from your landing page or within your project repository:

• up-to-date landing page
• all work done, accessible and navigable within your repository
• systems engineering governance: a requirements analysis, functional analysis, and/or system architecture as appropriate for your project. This should include evidence of how these have been iterated over time since your last audit.
• technical artefacts: prototypes, models, or progression of project outputs.
• progress against criterion specfied in the ConOps.
• test and evaluation plan or appropriate method for measuring value achieved in the project.
• plan for project completion and handover.
• any updates to the project management plan
• updates to project and WHS risk assessments if required.

Any updates to your ConOps should be undertaken with approval for your project host, tutor and team members between audits

Project Audit 4 (PA4)

Audit 4 is your opportunity to demonstrate your final deliverables. Your project should be wrapping up, but you may be permitted to explain why your project is not complete by the start of the audit due to project timelines. Your review will be explicitly against the expectations set out in your most recent ConOps. In addition to the previous Audit requirements, Audit 4 should include information navigable from your landing page or within your project repository:

• up-to-date landing page
• all work done, accessible and navigable within your repository.
• documentation of the outputs of the project e.g., testing, verfication and validation, documentation of the final solution, design or recommendation,
• handover documentation back to the project host and/or to the next project team.

Audit Grading: Progress Indicators

Grades are not assigned at audit milestones. Instead, teams receive progress indicators and feedback intended to help refine their processes and improve project outcomes.

Final group and individual grades will be determined through an ongoing, collaborative process throughout the year. The insights and feedback from Project Audit activities will inform the Final Team Review, which serves as the basis for your final grade. This will be determined in consultation with tutors and conveners. The following section outlines the formal evaluation process.

Projects will vary significantly between groups, depending on the scope of the project and the project host. Because of these differences, not all projects will progress at the same pace or produce similar outputs. These variations will not negatively or positively impact a project’s evaluation. Instead, we will assess each project holistically, making informed judgments based on the evidence presented.

Grades will not be assigned during the Project Audits, allowing teams the freedom to experiment, encounter challenges, and, most importantly, learn from the process. After each audit, teams will receive progress indicators aligned with the marking criteria, providing valuable feedback to guide their development.

The outcome of an audit will be based on the INCOSE guidance (modified to make sense in an educational setting) for milestones in the table below.

Do not proceed-continue this stage and repeat the review when ready

When awarded, these indicators are not fixed in stone. Your team may have an unacceptable early review, and turn the project around, and still do well. Engineering projects are an iterative process that develops and improve over time. The initial stages of the project do not define the whole project. As a guide, the ratio of consideration for Project Audits is 1:2:2:3, but each circumstance will be different.

These progress indicators will help to benchmark your final grade. Teams are expected to be fully autonomous in their benchmarking, so these indicators should provide one data point among others that you might be collecting.

Note: it is our experience that most teams naturally operate at or just above the ‘Baseline’ level. Work above this requires significant technical mastery coupled with extraordinary application of systems engineering, as described in the marking criteria below.

It is hard work!

Audit Progress Criteria

Progress indicators for this assessment task will be determined using the Many Eyes Process. The audit criteria and the audits at which each criterion is applied are summarised in the table below.

When evaluating a team against the following criteria, it is important to recognise that every team is different. This means that the factors considered under each criterion will differ between projects and may require some customisation. The audit process is designed to encourage teams to discuss and reach consensus as to what factors are appropriate at each stage of their project.

A description of expectations for each criterion against the indicator levels is outlined in the following sections.

Project Outputs

Limited to Outstanding

How valuable are the team’s outputs to key stakeholders (which extend beyond your project host), given the level of effort and other resources available to the team?

Decision Making

Disorganised to clearly traceable

How effective are the team’s processes for making, documenting, implementing, evaluating and learning from decisions?

Teamwork

Non-functioning to peak performance

How is the team working together to achieve project outcomes?

Stakeholder Engagement

Absent to productive and exceptionally coherent

How is the team communicating with, and managing the expectations of key stakeholders?

Reflection

Superficial to transformative

Note: not considered formally in PA1

How is the team reviewing feedback and acting on it to improve their performance?

Individual Audit Indicators

For each audit, individual students will receive two individual audit multipliers. The first, Team Review Multiplier will evaluate the feedback the student provides during the audit for both their own team and shadow team. The second, Team Member Contribution (TMC) multiplier will evaluate the relative contribution of a team member against the expectations of their teammates. The accumulative multipliers across the four audits are used to adjust the final individual grade for the project audits.

Team Member Contribution (TMC) weighting

During the Project Review process, each team member will provide peer feedback and evaluate the relative contributions of each team member. We would encourage you to use these opportunities to ensure you develop a strong and effective team. Students will self-evaluate, but this self-evaluation will mute in the consideration of the weighting.

Contributions will be awarded a weighting on a 5-point scale:

Note: No late submissions for TMCs are accepted. Failure to submit a TMC review will result in a -5% weighting as a penalty.

A TMC will be awarded for each Project Audit, and the result will be cumulative over the four Project Audits. TMCs will be awarded in consultation with your tutor and based on the reviews given through the TMC process.

Students should expect that all team members contribute equally to the project. If there are concerns about the contributions of individual team members, it is up to the groups to try to resolve these issues in the first instance. The TMC process, if approached constructively, should assist with identifying issues and working through them together. If–despite the process of giving and receiving feedback–you continue to experience issues within your team, you should contact the teaching team urgently to escalate your concerns, outlining what steps you have already undertaken. Team issues only raised at the end of the project will not result in increased multipliers for team members. Teams are expected to be pro-active in addressing and escalating issues and should not expect the teaching team to use TMC multipliers to “fix” an unfair situation at the end of the project.

It is not common practice to award high numbers of positively cumulative TMCs primarily because great leaders should be enabling other members of their team and improving the quality of project work (not seeking individual advantage).

The ideal situation is to have a well-functioning team with all team members’ contributions even (corresponding to all members being “at expectations”). If students in a group rate their whole team as “well above expectations”, there would be no advantage derived from this, as contributions are measured with respect to the team’s average. Quite the opposite, “flattening” peer evaluations at the high end of the feedback scale can make it hard for truly valuable contributions to be seen and recognised.

Project Review (PR) weighting

During each audit, team members will reflect on two assigned criteria from those listed in the audit progress criteria for their own project work, as well as the same criteria from their shadow team’s project. The criteria will be randomly assigned and vary for each audit. The published list of Project Audit Review Topics on Wattle will notify you of your specific responsibilities.

You will complete a Project Review form, which includes selecting progress indicators and providing a written critique of 200-250 words for each criterion for both your team and your shadow team (resulting in four written sections total). This process requires thoughtful investigation and analysis of the teams’ work as recorded in their repository. Your reviews should offer constructive, specific, actionable, and accurate feedback, adding value to both teams. Completing the review is an essential part of your contribution to the class as a collaborative and engaged participant.

You will be awarded a weighting based on a 5-point scale:

The Project Review weightings will be awarded for each review. It is not common to award high numbers of positively cumulative PR weightings, but there have been many extraordinary examples of students significantly improving the work of others through review.

Final Project Grade

Teams do not receive a grade through the project only progress indicators and feedback. At the end of the project these are used as evidence to apply for a grade based on the team’s performance across the project. The final grade for the group will be determined through an application prepared by the team, called the Final Team Review. This is submitted on the final day of the concluding semester for the project.

Final Team Review

Each team will provide an application detailing the team’s performance as follows:

• 3 pages (maximum) detailing performance against each of the five project audit criteria with reference to indicators (unacceptable, baseline, acceptable or exemplary).
• 2 pages of specific supporting evidence.
• A coherent overall grade band recommendation based on the Progress Indicators.

This will be considered alongside the progress indicators for each of the audits and, of course, the project repository and deliverables. You should develop this application in deep consultation with your tutor over the whole year. If this process is done well, you will have a clear idea of your group’s grade ahead of submission. Overstating, overshooting, or attempting to retrofit your project work to align with the audit criteria offers little benefit, as your tutor has been actively involved in your project journey and is familiar with your progress.

Determination of final audit grade for individuals

Multipliers from the four audits are used to adjust an individual student’s final audit grade. Your final individual weighting will be calculated through the following formula:

your group's project audit grade (based on Team Review submission) x (team member contribution (PA1+PA2+PA3+PA4) + project review multipliers (PA1+PA2+PA3+PA4))   

For example, a team member might receive the following weightings during reviews:

Example calculation

Let us assume that the team has had progress indicators throughout the semester in line with “Baseline” and “Acceptable with reservation” progress indicators, and successfully argue a Credit-Distinction grade in the Team Review. The Project is awarded a 67.5%. Therefore, the individual grade would be calculated as:

Project grade (67.5%) x Individual Weighting (0.95) = 64.125%

Another team member with an Individual Weighting of 1.1 due to exemplary Project Reviews and TMC would be calculated as:

Project grade (67.5%) x Individual Weighting (1.1) = 74.25%

Mid-Project Presentation (MPP)

Effectively communicating complex, detailed work to multidisciplinary audiences is a critical skill for professional engineers. This requires striking a balance between demonstrating expertise by providing sufficient technical detail and ensuring clarity to avoid confusing or alienating the audience. The mid-project presentation offers an excellent opportunity to practice and refine this essential skill.

By the midpoint of Capstone, your team should have developed a comprehensive understanding of your project, including its purpose and the value it provides to your project host and other stakeholders. You should also have a clear strategy for delivering a solution - or key components of it - within the course’s time constraints.

When preparing your mid-project presentation, approach it as a pitch to a diverse audience (which may or may not include engineers), aiming to convince them of the merit of continuing your project. This is not a marketing presentation but an opportunity to demonstrate your understanding of the problem your project addresses and the steps you are taking to resolve it.

Your presentation should outline the following:

• What is the problem? - What is the problem your project is solving? Who are the stakeholders? What is the value proposition of the project?
• Why is it important? - Who will the project impact? How will it impact them? What will the scale of the impact be?
• What is your solution and how did you decide on this solution?
• Why is this the right solution?
• How will you achieve this solution? - What is the approach you will use? What are the key challenges you will face and how will you mitigate these challenges?

Each team will be allocated 15 minutes for their presentation. This will be divided into 10 minutes of formal presentation by the team followed by 5 minutes of question time from the audience. Presentations will be attended by your fellow students and the teaching team. Project hosts are not invited on this occasion, but teams are encouraged to present to their project hosts at a time that is convenient for them.

Demonstrations of models or prototypes are encouraged; noting we understand many projects may not be at this stage just yet.

Your audience is the other engineers and professionals within the “Capstone Engineering Firm”. They will be technically minded people but may not be experts in your project’s field or familiar with the specifics of your project. Therefore, you should avoid using jargon and refrain from delving too deeply into the technical aspects of your project.

Think carefully about the audience for the presentation and its purpose when preparing the content. Note that material such as project plans, internal teamwork, governance, detailed systems engineering documentation and detailed technical design are too low level for this presentation. These are more appropriate to present within project audit 2.

MPP Grading

Multiple members of the teaching team will grade your presentation. Your work will be assessed against the following criteria:

• Appropriateness for audience: poor <-> suitable 25%
• Demonstration of the value of the project: ambiguous <-> clear 50%
• Professionalism, approach, and attitude: ad-hoc <-> considered 25%

For more information about the Mid-project presentation including a detailed rubric see Wattle.

Showcase Poster and Pitch

The Showcase is an exciting opportunity to share all your hard work with a wide audience.

The Capstone Showcase is an exciting opportunity to showcase your teams’ achievements towards the end of your concluding semester. At the Showcase, teams will present their work to an audience including project hosts, industry guests, academics, and other students with a short recorded pitch and a poster. The purpose of the showcase is to clearly explain to a lay audience what your project is about.

Skills modules on pitch preparation and posters will be run prior to the showcase.

Your poster and pitch should cover the following:

• They should clearly articulate the problem which your project aims to solve (goals of the project, project/stakeholder visions and objectives),
• Who are you solving the problem for (stakeholders)
• They should provide a summary of the outcomes and results you have achieved and how these have delivered value to your project host and other stakeholders.

The audience for your pitch will be diverse in their technical knowledge and experience.

Please check the content with your stakeholders as it will be made publicly available.

Pitch

• Your pitch should be a video a minimum of two minutes and a maximum of three minutes in length,
• Your pitch must be hosted via Youtube (only),
• Your recorded pitch will be accessible via a QR code on your poster but can also be included on your landing page.

Pitches can be presented during your tutorials in the week prior to the showcase to receive feedback from your tutor and shadow teams.

Poster

The poster will be set up on a pinboard as shown in Figure 1. You may bring props, prototypes, demonstrations, or other material to showcase your work.

Figure 1: Aerial view of layout for poster showcase

• Your posters will be printed for you at A1 size on premium paper; however, your poster should be designed in such a way that it could be printed at A2 (smaller), A0 (larger) size or be viewed online.
• There is no template that you should follow. This is an opportunity to be creative with how you present your project visually.
• Posters should be submitted in PDF format on Wattle, the Friday before the Showcase. The file should be submitted as a PDF at A1 size max 250 MB. Save your file to its highest resolution, use CMYK colours and embed your fonts to ensure they are not altered during printing.
• All posters should include the names of team members.
• Any logos or recognition from your Project host (including their names) should be endorsed by your project host.
• Your poster must be submitted to Wattle at the end of week 10 of your concluding semester to be printed in time for the Showcase event early in week 12.

Poster recommendations

• Think of your poster as a ‘conversation starter’ - it does not have to contain every morsel of information about your project.
• Avoid bitmap formats such as JPG, PNG, TIFF, and prefer scalable artwork, such as SVG.
• Where bitmap formats are used, the resolution should be high enough for printing (above 150 dpi at A1 size).
• Avoid semi-transparent backgrounds - these might look great on the screen, but generally don’t print very well.
• Maximise your poster for readability - with a minimum viewing distance of 2 metres, choose easily readable fonts. We suggest using a readable sans-serif font such as Public Sans, Helvetica or Roboto, and setting your minimum font size at 24pt. The ANU colour palette will keep it in theme.
• Some groups use a QR code to allow your audience to explore information later.
• Get feedback on your poster as you go - don’t save getting feedback until you’ve ‘finished’.

After the showcase, posters can be given to project hosts, or you might wish to take it home and hang it on your wall :) or you can donate it to the School of Engineering to inspire future Capstone students.

Posters from previous semesters can be accessed on Wattle.

Showcase grading

Your showcase will be graded against the following:

• Overall:
  • Content - Clear description of project goals, stakeholders, outcomes and the value delivered to your project host and stakeholders
  • Narrative and appropriateness for audience
• Pitch
  • Professionalism - quality of production and presentation
• Poster
  • Visual coherence including flow, readability, layout, use of space and use of visual aids

For a detailed rubric, see Wattle.

Professional Reflections

The Professional Reflections are independent activities that complement your ongoing professional engineering development.

As a professional Engineer you will be required to reflect on your skills and experiences to demonstrate your ability throughout your career. For example, when responding to selection criteria for a job application, seeking promotion or when applying for chartered status as a practicing engineer.

For your individual assessment in the Capstone Design Project, you will submit a professional rflection at the end of each semester (two submissions).

The first reflection will centre on your contributions and approach to the project during the initial semester of the Capstone. You will evaluate how you have applied and expanded upon technical and/or professional skills gained from other areas of your engineering education in your Capstone experience. Additionally, you will identify opportunities for further development one or more skills in alignment with the Engineering Stage 1 Competency Standard for Professional Engineers. This reflection will also serve as a foundation for setting personal and professional goals for the final semester of the Capstone project, enabling you to maximize your growth and impact within the team.

The second will reflect on the growth and development of your technical and professional skills through the Capstone project; how your approach to meeting challenges has changed through your degree and what that has taught you about your capacity to grow and improve as you prepare for graduation and the beginning of your professional career.

To assist with this process, it is recommended that students keep an Engineering logbook throughout Capstone, demonstrating their day-to-day work, host interactions, resources accessed, hours worked, ideas generated and personal reflections on how previous courses and extracurricular activities have assisted with the project. This can be used as evidence within your reflections. For more information see the section on keeping an engineering logbook.

Reflection format and grading

Both submissions will be a written professional reflection of 1500-2000 words (2-3 pages) and 2 pages of evidence.

Please see Wattle for more details and a marking rubric.

Engineering Logbook

A logbook is a progressive and cumulative demonstration of work done over the course by an individual student. It can also be described as a ‘technical diary’. It is much more than a log of time spent including information such as thoughts, ideas, sketches, calculations, results, contacts, references and more. This includes both documenting failures and successes. The practice of keeping a logbook often used in industry and research, and usually serves as evidence for:

• a time log accounting for how the project host “will be billed”
• a log of activities that were undertaken on the project
• a record of ideas during the project
• evidence of inventorship or origination of ideas
• a record of designs and innovation during the project
• a way to trace an idea through to realisation

Some organisations require engineers to keep a logbook. Many engineers keep a logbook as a personal preference. Regular entries into your logbook will allow you to track what you’re doing during the project and identify where you have used skills, tools, experience, and knowledge from other courses and/or extracurricular activities. Done well, this should be very useful when it comes to preparing your professional reflection and competency mapping.

A logbook is kept by an individual. Entries should be made regularly (weekly at a minimum) and must be timestamped (day of the week, date and year). You may choose to use a physical or digital logbook but should consider the advantages and disadvantages of each option before choosing.

The following resources may be helpful for beginning your logbook:

• OneNote -Engineering Notebook - How to use Microsoft OneNote to record a logbook
• The Value and Importance of keeping a logbook
• An investigation into the use and content of the engineer’s logbook - Available via the ANU library

Version Control

Acknowledgments

This course is an evolution of semester long course, ENGN4221 Systems Engineering Project. The current course convener Dr Zena Assaad acknowledges the immense support from Dr. Chris Browne for his contribution to build up and set up a fantastic Capstone program.

The original Course Guide for ENGN4221 Systems Engineering Project and associated frameworks were designed by Dr Chris Browne over many years of teaching, and ideas from this design have been published in multiple sources. Parts of this text originate from A/Prof Shayne Flint, from when this course was a part of TechLauncher. I acknowledge the hundreds of students, dozens of tutors and number of colleagues and project hosts who have provided input into the design as it is today. The design of the course today would not be possible without the critical collaborations with A/Prof Shayne Flint and Dr Lynette Johns-Boast, who pioneered authentic project-based learning in the Research School of Computer Science for over a decade, and Prof Richard Baker who showed me a better way to learn. - Chris Browne

Document information

Change log

2025.S1.01: 15/01/2025

• Creation of a restructured document for Sem 1 2025. Clarification of audit criterion and purpose of professional reflections.

2025.S1.02: 20/03/2025

• Completion of Showcase details and grading expectations

2025.S2.01: 04/07/2025

• Update email address