Formalisation, or the process of expressing problems in the notation of formal logic, is the most important skill students acquire in elementary logic courses, but it is also the hardest to learn and the one we teach least effectively. For several years at the ANU we have been using an online tool which presents students with problems expressed in English and allows them to encode these problems in a lightly enhanced version of first order logic. A back-end reasoning system generates solutions from the students' encodings, or reports that no solutions exist, and the students then iteratively debug their work until they get it right. Pedagogically, this is much more effective than pen-and-paper exercises, but it is still far from ideal: students spend too much time getting the syntax to the stage of passing the parser, and get little help with fixing semantic errors after that. They often report frustration with the site, despite its pedagogic virtues. The database storing all text that students have sent to the solver is by now a large and extremely detailed record of their activity on the site, which can be mined for information about how they approach problems, what techniques and strategies they use, where they get stuck and what gets them unstuck, when they change tack, when they give up and much more. We can observe these things at many levels, from the most abstract aggregated statistics to the most minute detail. For the first time, we have large-scale, reliable, accurate data on what students actually do when tackling a homework assignment. Moreover, the observations made do not perturb the system to any detectable extent. However, no tools exist yet to allow us to access this treasure. The first part of this project is to devise and implement such tools, using standard database retrieval, visualisation techniques and web programming. The second phase involves analysing the results, designing consequent improvements to the web site and implementing those. Finally, both a student survey and comparison of the data from before and after the changes will be used to assess the site upgrade.
1. Design and implement tools for analysis of the dataset. 2. With the supervisor, analyse the results from 1. 3. With the supervisor, design and implement improvements to the site in the light of 2. 4. Critically assess the results of 3.
This is an ideal project for a software engineering student with interests in web design, including HCI aspects, and in pedagogy. A background in logic would be a definite advantage. The formal requirements are those of COMP4540 or COMP4550, preferably with a Distinction or better in COMP2620. A part of this project could be undertaken as a summer internship. Contact me if interested.
Experience of a substantial project, at the leading edge of research into logic education. Opportunity for genuine impact on tertiary education in the discipline.