Wednesday, 18 September 2013

Water Under the Bridge - Reflection Time Again


Avon River punting station. L. Ainsworth March 2012

It seems like a long time since I reflected last. I have not maintained the intensity and excitement of the mOOC and scenario planning time, though the work we have done around the organisation has been extremely relevant and valuable for me at work right now and has informed some decisions and directions.
First, though, thoughts about the assignments. I took a few days away from course work and found it hard to go back. Work expanded into the gaps, though I did get a lovely weekend away at Le Bons Bay and a bit of time in the garden. I've learned (again) that it is best to keep at it with assignments and doggedly work to complete them; this piece of learning should help me with the final assignments and keep my motivation high in the next couple of weeks. I definitely appreciated the focused feedback provided when we received our grades, though it was not as detailed as some I had received in other courses. It was interesting to see where my assessment of my own likely grade using the rubric was out in a couple of places, but mostly quite accurate. By this stage of my career, I expect I should be able to assess my own outputs, so it was reassuring to find that was true.

Next, we looked at Wayne's analysis of the SP4Ed, and thought about the international context of our study. For me, the takeaway message was around the critical mass building the experience. I thought by the substance of the comments we were making that we all participated because we wanted to engage with the content and the people, not because the driver for some of us was the need to be there for the course credit. I would really like to go and read through the entire conversation stream again from start to finish. ( Not right now, though). Related to that was the interesting analysis of the source of our participation - I knew that I had migrated during the first week from Twitter to the WikiEducator messaging stream for most of my posts because the WE stream permitted slightly longer posts, and it gave instant gratification, rather than moving back and forth from my normal Twitter stream to follow the hashtag. Seemed like other people used that more as well and it would be interesting to know why. Who is there at different times of the day or night depended not just on the timezone but on what people had going on in the rest of their lives - work, family commitments, sport etc. but there was an international flavour most times I was there.

Working on e-learning benchmarking for the past two weeks has, as I said, been helpful. I have read a lot, as this was an area that was not familiar to me, though the general principles were, and I have used other models/evaluative tools to benchmark in other settings. Really engaging with what best practice can be for an organisation in general, and for mine and others in particular has been a learning opportunity that I am fortunate to get within the context of this course. Change with digital technologies must be measured if it is to be sustained and developed (this backs up reading that we did for the change models earlier in the course). The connection has become quite clear in the last week.
Ideally, I think, we should establish a base measurement before commencing each new aspect of change, but in the real world, does this actually happen in educational organisations? I would like to know the answer to that question. Do we always then close the loop, and build on the reflections and progress in the ideal action research spiral upwards? This fits with the four modes of reflection that are shown in a table in the reference below by Tiong. They are: reflection to action, reflection in action, reflection on action and reflection for action.
In my working life, I have not observed this happening in a consistent manner and maybe that is due to changes in staff, changes in priorities, changes in funding? Trying to model it as one person is difficult enough if wishing to achieve consistency!
Anyway, doing the practical analysis exercises has helped me with this particular section of the course - I do learn better if I reinforce my reading and thinking with some actual doing! Even the wretched mind-mapping was a good opportunity to move further forward with learning to use a new software than I managed in the earlier mindmapping exercise that we undertook. patience and persistence pays off.

Finally, when Wayne continues to remind us that everything we might want to know is quite likely available on the open web, (true e.g. last year I learned how to successfully patch an airbed from Youtube) spending time without power last week makes me very happy that I also have plenty of print resources available to me at home and at work. I miss being able to keyword search when using them, but I love that I can flick back and forth in the pages. However, I am glad that the storm wasn't closer to when our next assessments are due in.



References:
Rusten, E. & Shuman, S. (2008, June 21). Critical thinking in education. Retrieved from: http://www.slideshare.net/erusten/critical-thinking-in-education

Tiong, H.B. (2007, June18).Engaging in  reflective practice. [Blog post]. Retrieved from: http://www.classpointconsulting.com/articles/ReflectivePractice.html
Music listened to while preparing this blog post: This week it has been the CD Gloria: the sacred music of John Rutter. John Rutter has written some excellent Christmas carols, but this is some of his other work. I especially enjoy his version of  All things bright and beautiful.

Monday, 16 September 2013

Abridged eMM Evaluation


For this post I am looking at two eMM processes. The areas that they are based within are Development and Support. I am using the eMM 2.3 Core Assessment Workbook. This is not presented in the same manner as the eMM practices used in last week's analysis. It sorts the practices backwards into the dimensions of  optimisation, management, definition, planning and delivery. This confused me to start with when I began to analyse the relative strengths of the process capability dimensions.
The organisation that I have selected to consider for this task is a tertiary institution in the South Island of New Zealand. The programme being informally evaluated is representative of an undergraduate programme of study. The institution has a blended approach to teaching.



Process D7: e-learning resources are designed and managed to maximise reuse.
This is a significant process for the organisation, as it has only been developing such resources for around 3 years and establishing a base benchmark now will allow development to be monitored over the next stages of resource creation and reuse.
Overall, the institution is extremely variable on this with responses ranging from fully adequate through to not adequate. After comparison, it seems to be stronger in the Delivery area, with Planning second. Though intentions are good, more robust formal monitoring and documented policy is absent. Most of the documentary evidence for this particular process was not available to the public. In this process I also found one example where the practice in the institution has gone backwards.(incentives provided to teaching staff who create resuable e-learning resources).



Process S2: Students have access to a range of library resources and services when engaging in e-learning.
Coming from a library background, I regard the practices in this area as extremely significant when supporting the operational management of e-learning delivery. These areas are a more mature part of the organisation, but even so, there is scope for improvement. Definition was the area of most weakness followed by Optimisation.
One issue I have is around the assumption that the benchmark as established is actually best practice. In the past 6 years, I believe that practice has moved on, and the assumption that is made that a library will sit separately from the learning and teaching support aspects of the institution it supports is no longer fully valid. Some of these benchmarks also seem designed for very large libraries. Also, the assumption that best practice is to assign a designated library staff member whose contact information is made available is not, in my opinion, the fairest and most consistent approach to take for students. The institution evaluated here provides other means of contact that ensure students get an apt and timely response from the most appropriate staff.

Materials used to define the characteristic practices:
  • the website of both the institution and its library,
  • the website of the students' association
  • the course outlines and graduate profiles from the programme,
  • internally available material related to a recently developed project around digital objects.


Overall, I think the major point that this exercise makes clear is that mentioned in our course material: "The key point to note is that more e-mature organisations have evolved ecologies that ease the use and adoption of digital technologies. " (Davis, N. and Mackintosh, W., 2013,s.8). This organisation is not yet e-mature, though there are clear signs that it is proceeding in that direction based on the two processes selected for this exercise. The model permits a clearer vision on the areas where easing that use and adoption could be better achieved.

This assessment was conducted as a personal learning exercise to gain understanding of the eMM and my assessments were for the most part restricted to publically available information and limited to the two e-learning processes described.

Images:
The two images reproduced here are from the Openclipart  website. Images from this site are able to be freely reused.

References:
Davis, N. and Mackintosh, W. (2013). Section eight: The organisation. [Available from University of Canterbury Learn EDEM 630 web site].

Marshall, S.  (2007). eMM-Core self-assessment workbook. [draft report]. Retrieved from: http://www.utdc.vuw.ac.nz/research/emm/documents/versiontwothree/20070620eMMCoreWorkbook.pdf

Thursday, 12 September 2013

eMM Mindmap and mini-evaluation


This week, we are still thinking about the e-learning Maturity Model as described in the last post here.
This mind map is to identify the structural components of eMM and how they relate to each other. It can be viewed in more detail by clicking on the link in the caption.

Mind map constructed using  Mind Meister. (https://mm.tt/324710821?t=8QjvmyHl8P)



So what do I think is the value of the eMM for assessing organisational maturity of e-Learning in my personal context?

Strengths
Exploring the capability of the organisation is very important in the current tertiary environment. With many demands and strategies, the temptation could be to bite off more than can be chewed. By using a model, the organisation can assess where it wants to direct scarce resources.
The model has a comprehensive set of dimensions, fully based on past scholarly research and practice thus reducing the need to establish an assessment model for any one institution. Staff can spend the time conducting the assessment, not producing the assessment tools.
The model provides an ideal method of benchmarking performance amongst like institutions. The New Zealand Universities Academic Audit Unit ( now Academic Quality Agency for New Zealand Universities (AQA), Te Pokapū Kounga Mātauranga mō ngā Whare Wānanga o Aotearoa) have encouraged the universities to undertake relevant benchmarking activities, and this is extremely relevant to the quality assurance processes that might be expected in the cycle 5 academic activity themes.
The organisation can progress and grow through the levels provided. This is an active, not a static model, aiming to produce a more effective organisation. I like that it assesses normal practice not the high or low ends of evidence.

Weaknesses
This is a very long benchmark, and requires the appropriate amount of time to be set aside to complete it thoroughly.
So far, from a quick look at citations on Google, there do not seem to have been many published reviews of its use, other than by the original developers. This may simply be that institutions do not have time to write up what they are doing, but could be because not many places have adopted it in its entirety yet.
I think there could be a tendency for parts of an organisation to see more value in particular process dimensions. This is acknowledged in the notes on relationships between the dimensions.
I find it a little confusing that sometimes the practices swap between talking about a course to talking about the programme of which the course is a part. For institutions where courses are part of a number of different programmes, this could be a drawback.


Recommendation decision on whether you would consider using eMM as a for a more detailed assessment of e-Learning at an organisation of your choice later in the course.
 Yes, I would certainly be interested in doing this, however, the organisation of my choice currently adopts a blended approach to teaching, so there are few courses or programmes that are fully in an e-learning mode. Therefore, there might be times when I would need to make comments alongside the rankings. Also, I am a little worried about how much time it might take to do this later in the course. This activity was only supposed to take one hour and has taken me at least 4 hours. Sometimes a task that sounds initially straightforward to undertake is not actually so in practice. Having been introduced to the model here, I intend to use it anyway next year to conduct some analysis.

References:
 Academic Quality Agency for New Zealand Universities. (2013). Cycle 5 academic audit framework. Retrieved from: http://www.aqa.ac.nz/sites/all/files/AQA%20Cycle%205%20Framework%20Jan2013.pdf

 Marshall, S. (2006, November 9). So what is the eMM anyway? [Blog post]. Retrieved from e-Literate website: http://mfeldstein.com/so_what_is_the_emm_anyway/

Marshall, S. (2007). E-Learning Maturity Model: Process descriptions [draft report]. Retrieved from http://www.utdc.vuw.ac.nz/research/emm/documents/versiontwothree/20070620ProcessDescriptions.pdf

Music listened to while preparing this blog post: I am still on easy listening this week. As Kevin mentioned, not to everyone's taste, but when I struggle with mind maps I need to be soothed - this week it has been The Seekers, particularly the album The Seekers Seen in Green.

Thursday, 5 September 2013

Intuitive Assessment of part of an eMM Process

 Investigating the delivery dimension of the E-Learning Maturity Model.

The organisation:
For this intuitive analysis, I have selected a course from a tertiary institution where I have been a student. This course was offered with both an on campus and a distance option. I am analysing the distance option. Some of the practices listed in the model referred to the programme and not to an individual course, therefore it was a sometimes little difficult to rate the course as a single exemplar for the programme. In those instances I rated the course alone.

Intuitive Assessment Results for the Learning Process: (Codes - FA = Fully Adequate, LA = Largely Adequate, PA = Partially Adequate, NA = Not Adequate).

Course documentation includes a clear statement of learning objectives.
FA

Learning objectives are linked explicitly throughout learning and assessment activities using consistent language.
LA

Learning objectives are linked explicitly to wider programme or institutional objectives.
PA

Learning objectives support student outcomes beyond the recall of information.
LA

Course workload expectations and assessment tasks are consistent with course learning objectives.
PA

Courses provide a variety of mechanisms for interaction between staff and students.
FA

 Students are provided with teaching staff email addresses.
FA

 Students are provided with technical support for all of the communication channels in use.
PA

 Students are provided with explicit descriptions of the relationships between course components and activities.
LA

Courses include opportunities for students to practice with e-learning technologies and pedagogies.
PA

 Students are provided with e-learning skills support through a variety of communication channels.
PA

Course activities provide students with opportunities for substantive feedback on their e-learning skills.

PA


Value Judgement of my Reliability:
The analysis is based on my subjective memory of a course, it is not offered this semester and I have deliberately not gone back to check details since this exercise is to be intuitive only. It is based purely on my participation as a student and not as part of any team putting together or delivering the course. I have no access to student evaluations or reflections from the examiners or course designers, so I am not a reliable analyser in making this evaluation against the benchmark. However, my accuracy could still be reasonably high, just not based on sound principles.

Disclaimer:
Note that the above assessment is based only on my own memories and perceptions and has no evidential basis. It is compiled as part of formal coursework and is not to be used as valid assessment of any real course.

Music listened to while preparing this blog post: Songs from Peter Skellern's albums Sentimentally Yours and Right From the Start. And because I've finished this post and got curious I looked him up on YouTube and there are lots of recordings put up (not sure of the copyright implications of these) But here is a link to a YouTube link of Peter Skellern performing live - Still Magic. Then I discovered that he sang the theme song for Me and My Girl, a TV series I remember watching back in the 1980s. So now I am going to watch an episode and try to remember what I ever saw in it :-)