An evaluation of the Advanced Computing in the Humanities project (ACO*HUM)

A final report for submission to DGXXII European Commission

Prepared by European Economic Development Services Ltd for UNIVERSITY OF BERGEN
December 1998

EVALUATION OF THE ACO*HUM PROJECT

EXECUTIVE SUMMARY
1. Introduction
2. Description of the Project
3. Project Outcomes
4. Network Management
5. Innovation
6. Transnationality
7. The Future of the Thematic Network
8. Conclusions

EXECUTIVE SUMMARY

1. The Aco*Hum thematic network was established to develop effective coordinated mechanisms for integrating advanced computing into the teaching of undergraduate and graduate programmes across Europe.

2. The starting point of the project was that learning in the Humanities was being influenced by ICT and that teaching has to adapt to these changes. This stemmed from a recognition that computers alter the way people learn and this affects the way they would need to be taught.

3. The project concept - that teaching and learning in the Humanities needs an approach to the application of new technologies which is critically different from that used in other areas of study - is a useful one.

4. The project size was very large and represents a level of transnationality that has few parallels in Europe. It brought together a knowledge-base of combined expertise that can otherwise only be found in the United States.

5. However, from the outset the project was constrained by limited resources from both the SOCRATES/ERASMUS programme and partner universities. This immediately restricted what might be possible and led to a reappraisal of the project objectives.

6. Limited resources meant that partners who could/would not support academic staff input dropped out, thus limiting the base of active participation.

7. The first phase of the project, which began in the academic year 1996/97 included an exchange of information on the need for advanced computing in languages, history literature, art and other humanities subjects. It also involved exchanges of information on how institutions were addressing the teaching of new skills in advanced computing in traditional humanities disciplines.

8. The first year was taken up with the organisation of the network and awareness raising both within partner universities and between partners.

9. The second year had four main objectives:

broadening the base of active participation within a European context;
creating a high impact forum to present analyses of on-going work and promote discussion;
identifying best practice, and
disseminating initial findings.

10. As the project evolved some of the original aims were modified in response to institutional concerns and practical considerations. The six working groups which were established focused primarily on exchanging ideas and providing an overview for specific discipline areas - rather than on implementing proposals.

11. Network partners highlighted the following benefits from the project:

design of course modules, building on examples of good practice disseminated across the network;
adapting courseware for the www;
experiments in virtual mobility and communication between students and staff in the partner universities;
raising awareness in national educational institutions of developments in these fields;
raising institutional awareness of the importance of implementing technological innovation;
recognition of the benefits/opportunities from such a thematic network - in particular through collaboration across different humanities disciplines.

12. The project was perhaps initially over-ambitious in attempting to bring together 104 universities from across Europe. Some of the original aims had to be modified in response to institutional and practical considerations.

13. That the project had some successful outcomes was due to the individual commitment and enthusiasm from particular members of staff in several participating universities, and the early decision to have a three-tier management structure.

14. There is a need for universities to integrate computing technology with the teaching of humanities - so that they produce students better equipped to work in a society where integrated ICT skills are in great demand.

15. One major constraint was the inability of non-academic (ICT) institutions to access project funds. This limited participation by industry partners which is central to the employablity of students.

16. One benefit from the project was the cross-fertilisation of the computational linguistics project with other humanities disciplines.

17. Already the project has influenced changes in teaching and learning (in History) through the impact of ICT and the exchange of best practice in these areas.

18. However, many humanities students lack IT experience, and as well as subject specific applications there is a need for two core curriculum topics:

the use of IT ,and
ways of using ICT (which are going to change how humanities students will gather information and interact with the external environment)

19. At least two research fields were identified with potential in the Fifth Framework RTD programme - speech recognition/computational linguistics and the evaluation of bias-free search engines.

20. Particular projects were innovative in targeting new approaches to developing teaching material, and enabling students to acquire hybrid skills of value to industry.

21. The transnational element helped enrich the experience of partners, identify common problems and enhance cultural understanding, which in turn led to the development of multi-lingual projects.

22. The consensus from project partners was that the time scale of the project (two years) was too short to produce meaningful outcomes.

23. However, initial outcomes are sufficiently promising to suggest that further work is undertaken - in the form of

a SOCRATES bid - to take the thematic network forward, and
specific R&D bids through the EU Fifth Framework RTD Programme

1. Introduction

1.1 The objectives of this project were to establish a thematic network in Universities across the EU, focused on advanced computing in the Humanities - to promote joint curriculum development and to share and disseminate best practice in the development and teaching of the use of advanced computing techniques in the Humanities.

1.2 The primary aim of the thematic network was to develop effective coordinated mechanisms for integrating advanced computing in to the teaching of undergraduate and graduate programmes across Europe.

1.3 The starting point for the project was a perception that humanities students were increasingly being confronted by opportunities to acquire knowledge using ICT, and this has a direct impact on the way students need to be taught. This in turn requires changes in the syllabus/curriculum so that students can make the most of existing and emerging technologies.

1.4 As the project evolved it became clear that new technologies are having a different impact on different disciplines. In the humanities the standard commercial tools for word processing and image handling are inadequate for learning and teaching at university level. In teaching language and culture humanities scholars need refined computing tools which are able to handle sound and meaning, words and images, logic and art.

1.5 The project was led by the University of Bergen and began in the academic year 1996/97. It is one of 28 networks started by the SOCRATES/ERASMUS EU programme, and has now been in operation for over two years and involves 104 European universities within the network.

1.6 The evaluation is based on three sets of supporting information:

the interim and final reports of the Aco*Hum project and the 1998 Conference report - all prepared by the University of Bergen

responses from network partners to an e-mail questionnaire
a series of face to-face interviews with members of the project steering committee at the University of Bergen

1.7 The purpose of this evaluation is to assess how far the thematic network project has achieved its proposed aims and objectives, and to what extent it is meeting current needs in this field - including those being raised by other related projects and programmes.

1.8 The evaluation is also designed to address several aspects of European Community "Added Value" provided by this project. These include capacity building, innovation, transnationality, dissemination and impact on national/regional provision.

2. Description of the Project

2.1 The primary aim of establishing this thematic network was to develop effective coordinated mechanisms for integrating advanced computing in to the teaching of undergraduate and graduate programmes across Europe. This was to be done by identifying opportunities, problems and good practice and sharing information on these between network partners.

2.2 The first phase included an exchange of information on the need for advanced computing in languages, history, literature, art, and other humanities subjects. It also involved exchanges of information on how institutions were addressing the teaching of new skills in advanced computing in traditional humanities disciplines.

2.3 European collaboration was seen as a means of strengthening the advanced computing content within the general and subject specific humanities curricula - through the identification of learning modules and the means of delivery and learning. New processes of learning were explored, including collaboration with ODL projects and examining ways of increasing the accessibility of computational resources for humanities scholarship.

2.4 The management of the 104 - member network was initially organised through a three-tier structure with a Core Group (26%),an Associate Group (46%),and an Observatory Group (28%).However, as the project developed six working groups were established which reflected the most active project partners.

2.5 The first year was taken up with the organisation of the network and awareness raising both within partner universities and between partners. 2.6 The second year had four main objectives:

broadening the base of active participation within a European context
creating a high impact forum to present analyses of on-going work and promote discussion
identifying best practice and
disseminating initial findings

2.7 By the second year of the project six working groups were established covering:

formal methods in the Humanities;
computational linguistics and language engineering;
history and historical informatics;
computing in the history of art, architecture, and design;
computing for non-European languages;
textual scholarship and humanities computing.

2.8 At the end of the second year a major international conference was organised by the University of Bergen on behalf of the network partners. As well as disseminating information about the progress of the Aco*Hum project, the conference was used as a forum to break through discipline boundaries, national perspectives and to bridge research and educational practice. Transnational working, curriculum innovation, and course development on the Internet were all discussed.

2.9 Given the theme for the network it was appropriate that information and communication systems were used extensively to prepare for network and working group meetings and disseminate progress reports on the project. A web site was established (http://www.uib.no/acohum) and contains abstracts of conference papers as well as a record of meetings, reports, papers and surveys.

3. Project Outcomes

3.1 As the project evolved some of the original aims were found to be unrealistic and were modified in response to institutional concerns and practical considerations. For example, one original aim was the development of a Masters degree in Humanities Computing. However, after consultation with network partners it was felt that a more achievable objective would be to integrate computing within specific discipline areas. Similarly, the original aim of the network partners was to promote the participation of 1100+ undergraduate and 360+ postgraduate students and thereafter "a substantial expansion of participation as the universities within the ACH network benefit from the transfer of curriculum, teaching materials and experience". However, the overall limited EU resources and the focus on the Network at institutional level, meant that the active participation by students was not promoted.

3.2 The six working groups which were established focused primarily on exchanging ideas and providing an overview for specific discipline areas - rather than on implementing proposals. The project also had a slight change in emphasis when it was felt by the network that the development of ICT should not focus on technical areas but should be integral in linking Humanities with the IT industry. It was hoped that the presence of the project would encourage universities to provide a greater allocation of ICT resources in the humanities, and would boost research and the employability of this research in ICT. Some mapping of this was carried out within discipline areas - although no strategic outcomes were put forward.

3.3 In the first year of the project the network was established and sustained largely through individual enthusiasm rather than institutional commitment. There is a need to secure institutional support if the project is to have a long-term future, and there are encouraging signs that this is beginning to emerge.

3.4 One constraint on the project was that although it involved joint working between universities and non academic (ICT) institutions, the latter could not access project funds. This limited the participation by industry partners - yet their involvement is essential if the project is to go forward. It is also central to the issue of employability of students.

3.5 Although the network did not promote the active involvement of students, for one working group - the History group (a joint project between the University of Bergen and University College London) did involve students. However, many of the students lacked IT experience and it became clear that the curriculum needed to cover a general introduction to IT - followed by tailor-made, market-related ICT courses.

3.6 This focused on a core area of the project how can EU Arts faculties develop a common strategy to develop both generic and subject specific IT applications in the Arts, and how can these be embedded in the curricula?

3.7 The History group experience emphasised the impact of ICT in introducing different ways of teaching and learning. Teaching has changed dramatically because of the application of IT. At the University of Bergen this has led to the restructuring of the whole system of teaching history. At the network level the project has made an important difference - in terms of other EU University History departments exchanging best practice in the applications of IT to teaching. Because the Norwegian University system gives a degree of autonomy to departments these changes can be implemented fairly quickly. In other EU countries because curriculum content is more centrally controlled, such changes will emerge more slowly. A positive outcome was the resourcing of a new teaching post in History to develop material for teaching delivered via the web site. In the Netherlands the University of Groningen are taking a similar approach.

3.8 One other aim of the project was to consider ways of providing actual and "virtual" mobility for students. This was considered within the subject area of computational linguistics. This group set up 6 web-based pilot projects - dealing with the delivery of teaching materials. However, they found the overall integration into the curriculum of new methods was more difficult than anticipated. The aim was also to improve the quality of teaching materials in this field. One benefit from the network was the cross-fertilisation of the computational linguistics project with other disciplines.

3.9 Another important outcome of the project was that active co-operation was established between the Computational Linguistics working group and a separate SOCRATES thematic network project focused on Speech Communication Sciences.

3.10 Responses from other Network partners highlighted the following benefits from the project:

design of course modules, building on examples of good practice disseminated across the network;
adapting courseware for the www;
experiments in virtual mobility and communication between students and staff in the partner universities;
raising awareness in national educational institutions of developments in these fields;
raising institutional awareness of the importance of implementing technological innovation;
the 1998 Bergen conference made all the network partners realise the need for and possibilities/benefits from such a thematic network - in particular the importance of collaboration across different Humanities disciplines. There was a clear awareness of the value of disseminating ideas which, although developed for one area of the humanities could be adapted for other disciplines.

3.11 On the downside, several respondents from the Thematic Network felt that it was given extremely limited resources from the European Union. This had several consequences. First some of the original network partners were unable to continue with the project. Secondly, there was limited institutional commitment to the project;and thirdly, the project developed at all was largely due to the commitment and enthusiasm of individual academic staff in the partners universities.

Many felt that if this initiative was to be sustained it required some degree of institutional commitment - which indeed is beginning to emerge.

3.12 One broad strategic outcome was the identification of nodes for research initiatives and resources for applications of advanced computing in the humanities. For example, in the field of non-European languages five outcomes were envisaged:

an EU consortium with external funding.
lessons/feedback from an applied African Linguistics Masters.
lessons from setting up an MA on cultural lines.
establishing resource notes or a research node*
establish a model text corpus/database for training students.
evaluation of bias-free multi lingual search engines*

3.13 The working group see these as future actions, provided funding can be obtained.

* Note: These could form the basis of bids for Research and Development funding within the European Union Fifth Framework Programme for Research, Technical Development and Demonstration activities 1998-2002

4. Network Management

4.1 The first challenge of the project was to effectively manage 104 universities. This was done initially by sharing of knowledge and identifying those potentially active partners who would form the project core group. The successful early decision was to organise the network into three groups:

a core group (26%) committed to 30 full-time days per year.
an associate group (46%) giving 10 working days per year.
an observatory group (28%) committed to 5 working days per year.

4.2 This concept worked well, but it quickly became apparent that within the core group a number of subject-specific working groups were needed. These consisted of the most active and committed partners and helped overcome the initial feeling of isolation among specialists in their own institutions.

4.3 In the first year four working groups were set up (see section 2.7) and in the second year computing for non-European languages and formal methods in the humanities were added. Two of the six groups (Textual scholarship and humanities computing and Formal methods in the humanities) have a cross-disciplinary remit and take a broader humanities-wide perspective.

4.4 The working group on formal methods was asked to address three particular themes

how does the nature of humanities scholarship change due to the incorporation of formal and computational methods ?
what are the theoretical and practical problems involved in the integration of advanced computing in humanities curricula, across the various disciplines involved?
In which respects must humanities computing be taught differently than other computing?

4.5 Periodic meetings of the working groups have produced a number of benefits:

they have helped cross-fertilisation of ideas;
they have been able to deal with smaller, more manageable parts of a very large project;
deep structural differences between humanities fields were identified and brought into the open. This forced people to address these and to realise the need for consensus building, and to have common mapping of issues;
many common issues/concerns were identified, and these will now be looked at by a methodology working group.

4.6 The real role of the working groups was that they are made up of "informed end-users"; that is they know what they require to apply advanced computing in the humanities - and they can provide clearer specifications for the technical producers.

4.7 One criticism of the network has been a limited amount of feedback from the individual working groups to the steering group, with some communication problems emerging. There is an organisational issue in the tension between the working groups and the steering group and the ownership of development proposals. The challenge for the future is to improve lines of communication between the working groups and the steering group.

4.8 The most effective means of communication have been the web site and electronic mail. The meeting between the working groups at Granada was also very effective.

5. Innovation

5.1 Innovation is generated by new ideas and the exchange of ideas and information. In this project there were several activities which encouraged innovation. These included:

innovation in course design
innovations in teaching and learning
the general application of computing in the humanities
the recognition by researchers and educators of the European dimension in both education and their specific area of activity. (This is in itself an innovation).

5.2 Other innovations included spin-offs from the working groups, for example:

mobilising academic partners for self-assessment SWOT analysis;
real innovation ie, doing new things in a new way;
students acquiring hybrid skills from a cross-disciplinary approach to teaching and learning;
addressing most of the hardware/software needs in the humanities.

5.3 Particular projects such as those in History or Computational Linguistics were also innovative in targeting new approaches to developing teaching material (History) or enabling students to acquire hybrid skills of value to industry (Computational Linguistics). (This latter project has real potential as a Fifth Framework RTD bid)

5.4 In the University of Bergen the teaching of History has changed dramatically because of the application of IT - and has led to the restructuring of the whole system of teaching history.

5.5 The meetings of the working groups led to new ways of thinking in that they cross-fertilised ideas between different humanities disciplines and led to recognition of common problems and issues which demanded common solutions.

5.6 Perhaps the most significant overall innovation was the transnational aspect of the project which provided an added dimension to each working group and which encouraged the exchange of information and ideas.

6. Transnationality

6.1 With 104 university partners from 15 Member States of the EU, plus 2 associated countries of the EU, this project represents a level of transnationality that has few parallels in Europe, and has brought together a knowledge-base of combined expertise that can only be found in the United States. This network has begun to address the whole issue of the multi lingual aspects of computing use. It was quickly realised that transnational working would only be effective through pooling of resources, know how and expertise.

6.2 However, transnational working requires time, and a common comment from project partners was that the time scale of the project was too short to produce meaningful outcomes.

6.3 The transnational element worked well at several levels:

through the overall management of the network, and
the operation and composition of the working groups;
through the pooling of resources to achieve the objectives of the project;
it highlighted the multi-lingual aspect of the Aco*Hum project;
it provided a means for cultural transference within languages engineering.

6.4 The steering group felt that transnational working enriched the experience of partners and helped identify common problems. It also enhanced cultural understanding and the development of multi-lingual projects - around the issues stemming from multi-lingual communication

6.5 Most curricula are aimed at the respective national markets. The Aco*Hum network realised that competitiveness in curricula development is gained by having/fostering a transnational/international outlook.

6.6 The respective network partners/working groups promised country profiles - but these were not delivered - because of resource constraints and organisational structure. It was not found possible to have a suitable geographic distribution in the working groups because not all countries were active participants in the project. Each working group only had about 5 active partners, and some countries were only members of the observer group

6.7 It was generally felt that limited resources affected transnational developments - in dealing with 17 countries across the EU and in attempting to cover all issues raised by the working groups. Some countries were totally under-represented because of resource constraints. This also restricted the production of the thematic network database.

6.8 Moreover, there was a lack of synchronisation between the development of the project and other emerging EU funding initiatives - for example such as funding for the Eastern European accession countries. Again lack of resources (and time) meant that it was not possible to establish an extension of the Aco*Hum network within these countries. However, it is clear that Central and Eastern European countries should be included in any future project.

7. The Future of the Thematic Network

7.1 Each working group was asked to identify one pilot project for the future, with an overall focus on teaching and research training. It was also felt that the business applications potential of these developments should be recognised - if support for future development was to be sought from governmental and business decision-makers.

7.2 it was generally felt that the network needs to be in existence for a further 1 to 2 years to draw fuller conclusions from its operations.

7.3 The steering group foresee a number of options for the future. These are:

Aco*Hum devolves into several smaller networks;
Aco*Hum devolves into several consortia, but retains its overall integrity;
it splits into 6 smaller projects based on the existing working groups;
it evolves into a European Association;
it becomes integrated into the activities of an existing association (such as the Association for Linguistic and Literary Computing);
Aco*Hum is established as a European Union "Observatory" covering computing in the Humanities.

8. Conclusions

8.1 The project was perhaps initially over-ambitious in attempting to bring together 104 universities from across Europe. Some of the original aims had to be modified in response to institutional concerns and practical considerations. For example, the development of a Masters degree in Humanities Computing and the participation of over 360 postgraduate and over 1000 undergraduate students were not followed through.

8.2 Resource constraints restricted the number of universities who were active partners in the network, as well as the number of project outcomes.

8.3 That the project had some successful outcomes was due to two factors:

first the individual commitment and enthusiasm from particular members of staff in several participating universities
secondly the early decision to have a three-tier structure for the management of the network

8.4 There is a need for greater institutional commitment to projects of this kind, if they are to succeed in the long term.

8.5 European institutions of higher education need to agree a policy of integrating computing technology with the teaching of humanities - so that they produce students better equipped to work in a society where integrated ICT skills are in greater demand.

8.6 It was hoped that the presence of the project would encourage universities to provide a greater allocation of ICT resources in the humanities, and lead to greater links between Humanities and the IT industry. It is too early in the life of the project to say whether this aim will be realised, although there are one or two encouraging signs.

8.7 One constraint on the project was the inability of non-academic (ICT) institutions to access project funds. This limited the participation by industry partners, which is essential if the project is to go forward, and is central to the employability of students.

8.8 One benefit from the project was the cross-fertilisation of the computational linguistics project with other humanities disciplines.

8.9 Only two years after the start of the project, the experience of one working group has already led to changes in teaching and learning through the impact of ICT, and the exchange of best practice in these areas.

8.10 Many Humanities students lack IT experience, and it quickly became clear that the curriculum needed to cover a general introduction to IT - followed by tailor-made ICT courses for particular Humanities disciplines.

8.11 One broad strategic outcome was the identification of nodes for research activities, at least two of which - speech recognition/computational linguistics and the evaluation of bias-free multi-lingual search engines - have real potential within the Fifth Framework RTD Programme.

8.12 The most effective work of the project resulted from the activities of the six working groups which were set up.

8.13 The project helped raise awareness in national educational institutions of developments in the applications of ICT in the Humanities.

8.14 The 1998 conference made the thematic network partners realise the importance of collaboration across different humanities disciplines;and the value of disseminating ideas, which although developed for one subject area, could be adapted for other humanities subjects.

8.15 Periodic meetings of the working groups have helped cross fertilisation of ideas and to build a consensus across different discipline areas.

8.16 As the working groups consisted of subject specific expert practitioners familiar with the applications of ICT, they can identify and provide clearer specifications for the producers of the technology.

8.17 One criticism of the network has been the limited dialogue between the individual working groups and the steering group. This has led to tensions around the ownership of development proposals.

8.18 Given the theme for the network it is appropriate that the most effective means of communication have been the web site and electronic mail. These were also used to prepare for meetings and disseminate progress reports.

8.19 The project led to innovations in course design and students acquiring hybrid skills from a cross-disciplinary approach to teaching and learning.

8.20 Particular projects were innovative in targeting new approaches to developing teaching material (History),or enabling students to acquire hybrid skills of value to industry (Computational Linguistics).

8.21 Perhaps the most significant overall innovation was the transnational aspect of the project, which provided an added dimension to each working group and which encouraged the exchange of best practice, information and ideas.

8.22 With 104 university partners from the 15 member States of the EU, plus 2 associated countries, this project represents a level of transnationality that has few if any parallels in Europe, and a knowledge-base of expertise that is unlikely to exist outside of the United States.

8.23 The transnational element worked well through the overall network management and the operation and composition of the working groups, as well as the pooling of resources and expertise to achieve the objectives of the project.

8.24 Transnationality also helped enrich the experience of the partners, identify common problems and enhance cultural understanding - which in turn led to the development of multi-lingual projects through addressing the issues stemming from multi-lingual communication.

8.25 The Aco*Hum thematic network realised that competitiveness in curricula development is gained by fostering a transnational/international outlook.

8.26 However, transnational working requires time and resources. A common comment from project partners the time scale of the project (two years) was too short to produce meaningful outcomes;and some countries were under-represented in the working groups because of resource constraints.

8.27 However, the initial outcomes are sufficiently promising to suggest that further work is carried out to take the thematic network forward through a further SOCRATES bid, and to identify promising projects for funding bids through the Fifth Framework programme.