Project Audits (PA)
Project Audits are used to evaluate the progress and governance of each team
Purpose
Project Audits are designed to provide formative feedback to guide your group towards your client’s project goals whilst demonstrating high-quality project governance. Project Audits are standard practice in many industries to evaluate the progress of a project.[^1]
Goal
To enhance all project work through constructive, actionable feedback with critical and considered review.
Overview
There are three Project Audits during semester to support and guide your project:
- Silent Project Audit 1 - Week 3, to set or reset the agenda and scope for the semester
- Project Audit 2 - Week 6, to guide and evaluate progress, based on the project scope
- Project Audit 3 - Week 10, to finalise and prepare for the next stage of the project
Requirements
Each project will be different, and all aspects of the project should be negotiated with your client and TechLauncher staff
27 August 2024 Do not use WeChat for team communications. Some examiners and program facilitators cannot be on that platform, and therefore cannot properly assess your Teamwork and Stakeholder Engagement if your team uses it. Please ensure transcripts of team communications are available to all stakeholders, including your tutor and shadows. Help us help you! The University is on the Microsoft ecosystem, which includes Teams. In in doubt, use that.
Before you attend an audit, or indeed progress a project, make sure you have read our notes.
Each Audit is conducted over the course of a week. During tutorial sessions each team will present their work to their tutorial class and answer questions from those in attendance. There are no specific requirements for these sessions, and they are not directly assessed. The structure and timing of these sessions will be negotiated with your tutor. Note however, that these sessions are very important as they can be used to guide those who will be reviewing your work, and to help them understand what you have done. This will, in turn, help your reviewers complete a fair review of your work.
During each audit week, the “Many Eyes Process” will use detailed inspections of material stored in project repositories to evaluate the actual work completed by each team.
Audit 1 - Silent Audit - Requirements
The aim of Project Audit 1 is to demonstrate that your team is prepared for the coming semester. You should work with your client and tutor to determine applicable details such as:
- the setup of tooling for development, management of tasks, and project repository
- create a Team Charter
- create a Statement of Work (SoW)
- SoW should have project milestones, scheduling and deliverables for the semester
- SoW should identify technical and other constraints (eg. reliability, security, safety)
- Sow should identify resources, risks, potential costs and who will bear them
- SoW should be signed/initialed by your client
- SoW should articulate your client’s vision and objectives in your own langauge
- SoW should show evidence of stakeholder analysis
Continuing Teams should review and revise the above details to ensure that they are ready to continue their project during the coming semester.
Later Project Audit Requirements
The aim of the Project Audits is to demonstrate that your team has been working effectively and delivering real value to your clients, to each other, and to shadows. Reviewers will be looking for evidence and quality of:
- effective and appropriate decision making
- meaningful and useful project governance: decision log, risk register, and reflection log
- signed record of formal acceptance criteria with client
- roles and activities related to the delivery/acceptance process
- quantitative record of progress towards acceptance/delivery
- value delivered to the client
- appropriate lean and fit for purpose documentation
- technical artefacts including, as applicable, models, source code, test data and prototypes
If you are spending over 15% of effort on governance, that is at the limit. Governance should be between 10-15%, or 20% if you are finding your feed, to total effort. Note, nowhere above or on this page have we mentioned “slides”… Advise from David R.: “Audits are a point-in-time assessment of the team’s progress, and should ideally be a guided tour through the repository, covering the project status, team process, forward plans, risks and issues, etc. Showy (PowerPoint) slides consume time that could be used more effectively.”. To be clear, we do not mark or assess slide decks in TechLauncher.
Repositories
Each team is free to make their own decisions regarding the tools they use and where they store and manage the artefacts they produce. However, in making these decisions, each team should note the following:
-
Consultation with Client. Each team should discuss the choice of tools and repositories with their client. In particular, teams should discuss IP issues, who needs to access to what, how access will be managed and who will cover any costs (eg. GitHub, Jira fees).
-
Access. In addition to any client access requirements, your repository should be accessible by your shadows, tutor and course convenors.
-
Project Transition arrangements. Most TechLauncher projects will continue in some form after the course ends. When choosing tools and repositories, you should consider the needs of the next TechLauncher team or your client. For example, you need to ensure that they will have access to, and control over, any tools you have used.
Audit Landing Page
Each team is required to submit a link to a page, file or other document that guides the reviewers through the team’s work. As a minimum, this guidance should include links to the documentation and technical artefacts you want reviewers to look at during the audits.
The Audit Landing Page can be located on any of the repositories you are using (eg. gitlab, github, Google drive, SharePoint)
Teams should ensure that all reviewers have access to the landing page and the artefacts it refers to. If applicable, the landing page should provide instructions on how to gain access to the repositories and tools you are using (eg. how to get accounts, passwords etc.)
Note that the Audit Landing Page should not contain any of your actual project work - this will already be stored in the tools and repositories you have been using. Remember that an Audit is a look at your progress at a specific point in time, and does not require you to do any additional project work just for the Audit.
Teams using ANU GitLab or another Git repository should consider using the readme markdown file in the root directory of their repository as their landing page.
Evaluation Criteria
Marks for this assessment task will be determined via the “Many Eyes Process”. Marks are based on evaluations and feedback received from tutors, clients, shadows, and team members themselves. When evaluating a team against the following criteria, it is important to recognise that every team is different. This means that the factors considered under each criteria will differ. The audit process is designed to encourage teams to discuss and reach consensus as to what factors are appropriate at each stage of the project.
Input to this process will be Tag Reports that address the following criteria:
- Outputs [limited outstanding]
- How valuable are the team’s outputs to key stakeholders, given the level of effort and other resources available to the team?
- Decision Making [chaotic structured and efficient]
- How are the team’s processes for making, implementing, evaluating and learning from decisions?
- Teamwork [uncoordinated highly skilled coordination]
- How is the team working together to achieve project outcomes?
- Stakeholder Engagement [absent productive and exceptionally coherent]
- How is the team communicating with, and managing the expectations of key stakeholders?
- Reflection [superficial insightful and driving positive change]
- How is the team reviewing feedback and acting on it to improve their performance?
Evaluation Process
Projects will be evaluated using the Many Eyes Process.
Item | Team Members (self) | Shadows | Tutor | Client | Conveners |
---|---|---|---|---|---|
Tag Reports | If required | ||||
Quality Control | If required | ||||
Team Member Contribution | If required | If required | If required |
Submission
Outputs should be uploaded/updated progressively to repositories to demonstrate incremental progress by each team member. ‘Big Bang’ updates the morning of an audit do not demonstrate effective progress, and there is limited value in generating artefacts at the ‘last minute’ prior to audits.
Each team must maintain an up-to-date and valid link to their Audit Landing Page on a circulated google sheet by 0900 Monday in the Audit Week. A team spokesperson shall be provided write access to that sheet. It is the team’s responsibility to ensure that access to project artefacts is possible, and to keep the project records up-to-date on a weekly basis.
[^1]Software Engineering Body of Knowledge (SWEBOK) (Chapter 10, Section 2.3) provides a good overview of why project reviews and audits improve the technical aspects of projects.