Saturday, 31 March 2012

What should we be assessing?

We assess the wrong things. The students came to one of my lecture after an MCQ and they had been told that they cannot use anything but Blackboard during the test. This is reasonable as if they could use the internet they would find all the answers from Google and Wikipedia. But in a real life environment and not an artificial educational one you would have Google and Wikipedia so why do we need to memorise those facts? All of our subjects are too large to hold in memory any more and so you have to rely on books or the internet. So why are we assessing something that will be irrelevant to the future lives of students. 

I am setting an online statistics test and I have taught them how to work out statistics with a calculator just using sums, squares and square roots. But this was the first time in my life I had ever done it this way. I always use software to do it because the chances of making a mistake with 40 numbers is too high. I was going through the mock test where you have to calculate 30 numbers and one of the students said but why not do it in Excel? Well why not? I had expected they might think of this as I had used it to create my mock answer sheet! The test is online so they will have a computer with Excel so why shouldn't they use the skills they have? So my assessment has to make sure that those that know Excel do not have a significant advantage over those who don't or it tests their understanding of how you use statistics and not the process of carrying it out. The way I have designed it now it can be done as shown in the lectures, in Excel, by calculator ... So I let students think outside the box and use what they know. If they learn SPSS - a better stats programme to do the exam then even better. They have constructed their solutions to the problems. I have not made them unlearn what they know or made the assessment more artificial than necessary.

Friday, 30 March 2012

What should education do?

 What are the ultimate learning objectives of education? Now that students are having to pay they will be more focussed on getting something out of their university studies rather than studying for pleasure. 

What we all want is a good quality of life. We use education to get to that quality. So it is most often about career as this provides the money and this provides the food, the shelter ... The Experience Designer proposes that education is a response to try and meet the needs of the current society. The problem is that this is not very adaptive and because changing the way we educate takes so long we are always teaching what was needed 10 or 20 years ago and not what is needed now. We try to teach better but employers tell us that graduates are worse and worse and meet their needs less and less.

The other problem with our current education system is how wasteful it is from a Constructivist perspective. Everyone has a unique background. Every has different skills/competencies how ever you want to call them. Also everyone has had different experiences. I was just marking first year students CMS exercises and they too often focus on what they learn in formal education and forget that they learn soft skills in their everyday lives. We are influenced by what we read, what we choose to read, what we watch, who we talk to and all of this is then lost when we try to teach students one way and one set of solutions to a set of questions. Students should be thinking about their studies in the context of their lives and wider experiences but they are not. The ones that do get it - those are the ones who got the high marks for the assignment.

Thursday, 29 March 2012

E-learning collaboration

We cannot read the tutor's minds but they have managed to show us the strengths and weanesses of collaborative tools while we have been within the collaborative experiment itself. That is quite an acheievment. My head is buzzing around with so many views that this is going to be a bit random as I an struggling to get a clear picture of the lessons.

There are problems with various collaborative tools in that they fail to actually create a fully functioning dialogue. The point of Web2.0 technologies is to make it a read-write web, but often it just becomes a different way of authoring and viewing, there is reading or writing but not both. This would not meet Laurillard's objectives in e-learning to have a dialogue as either one side or the other is speaking but there is no fusion of the discussion. This can be a problem with asynchronous discussion boards. Each person makes posts but they never build into a conversation. This is why the tutor always plays a significant role in bringing the ideas together as suggested by Gilly Salmon. There is a T-shirt that says more people have read this T-shirt than have read your blog. A blog without comments is not a conversation - it is a monologue and so there is no opportunity to develop ideas.

Most of us felt best about Skype where there is a stronger feel that you are involved in a conversation because the responses are real-time. But does this give time for reasoned response and were the discussions more than superficial? Can you construct new knowledge at that pace?

Some of the discussion boards worked well, but we had one example that failed miserably and this is an important lesson. There are many different dynamics in a group and you can never be sure that one method will work for everyone in a group and in some cases it may not work at all. Even in the second exercise when we looked at types of learner people have only partially engaged with the technology. We have commented on each others posts but there is another chance for reflection by rating posts and most of us have been too polite to enter a rating.

We also looked at Wikis. The reason why wikipedia works and why they are worried is that it depends on having a large community of editors. They might not agree but they will keep writing and rewriting and so the hope is that over time articles improve. In this case we did not have this degree of collaboration because we did not edit each others posts. It is that read-write dynamic that leads to changed perceptions and learning. Like with a blog without comments a wiki without edits is another monologue and there is little deep learning, or more significantly advancement of the ideas.

Where most of the discussions have taken place and most of the collaboration has happened is in the personal blogs through the commenting. Here we have had the most active discussions even though these are the places where we are writing something personal that is not per se intended to be collaborative. 

So how would I sum it up? Sometimes you get collaboration where you were not expecting to find it and sometimes there is little collaboration on a task you designed to stimulate it. People and especially groups of people are very difficult to plan for.

Wednesday, 21 March 2012

Assessment for Learning

 am just reading The Experience Designer. This is a bit of an alternative view of learning that puts learning in the context of something we do all the time in order to understand the world around us and to buffer our personal world against the confluence of modern life (if he says that once more I will scream). This means that learning is the construction of a narrative. So this is a bit of a terrifying view. To accept this view we would have to make a big step in education to make it less reactive and more adaptive, but this might be too big a change to make and risks the careers of all those who are students during the change. The book sets out to show how e-learning can make this change happen.
I can see the good points but I was reading reviews of the book on Amazon. The first person loved it and thought that it was a paradigm changing book for e-learning. The second said that it was a nice idea but just not practical or realistic. According to that reviewer the critical thing is that we need to assess more to see that learning is happening, but we do need to get away from standardised tests.
Assess more! We have just been reviewing our courses to reduce the amount of assessments and they are suggesting to use more. Of all the things we hate as academics marking must be the most hated part. I am going to use more online assessment for formative feedback during my courses - especially statistics as the students have been struggling with not having more examples that they can do themselves. My perhaps diffident reply is go and find some examples in a textbook - those things we have in the library, the big rooms with books in them. As they are supposed to be independent learners, but they seem to want more hand-holding.

Prezi - this is so cool

Ok now I am addicted to Presi. I heard about it through BBC Click and I had seen it before as sometimes my wife uses them for her secondary school teaching but this is just great. It is so good that we can have a free private educational account because we have an educational e-mail address but I think the university should pay for some support as this is a service we want to grow and grow. These are just two examples.

Tuesday, 20 March 2012

Defensive Teaching

Last week we had a talk about students taking legal action against the university, which terrified lots of my colleagues. I have been lucky enough at my previous work to have three students take appeals against my course and two of them made it all the way to the Office of the Independent Adjudicator. Such cases are usually rare - in my case my cohort of MSc students was about 120 over the 5 years and I had 3 difficult students, so I have had more than my fair share.
I was thinking about these people in the context of the saboteur. These were students who came angry at the system and who thought that it should reward them in the way they wanted. They were paying £20,000, they had made it to this elite institution so they were brilliant and so they were going to get the brilliant marks they expected. Otherwise the system was letting them down. It was not their fault in anyway (I am reminded of the genetically perfect failures in GATACA).
  1. Student A: Did not get the marks he expected. He appealed and the external examiners remarked them and agreed with the given mark. He then found the name of the externals and checked their cv and wrote to them saying that they were unfit to judge his work. Eventually after he exhausted the appeals process he went to the OIA and they dismissed it almost instantly.
  2. Student B: Had a bad experience with his old university and appealed every mark he was given. When we eventually had enough and did not allow him to progress he appealed and accused the university of corruption and of selling degrees. Wisely he did not take that to the OIA but I had over 6 months of internal appeals.
  3. Student C: Was my project student. Projects are marked by supervisors and another marker. I read his draft and told him it would fail and how to change it. He ignored my advice and submitted it as it was. It failed. The external examiners agreed the fail. In the exam meeting a wrong mark was put on the sheet. This was crossed out and the correct mark entered, but he was sent this original where you could make out the crossed out mark. He appealed to the OIA claiming corruption and discrimination. They rejected his claim after consideration of the evidence, the key was having all the minutes and documents.
This kind of destructive student is a child having a tantrum and throwing their toys. In an online environment these sorts of postings with petty incriminations can destroy group cohesiveness. Even worse are the chinese whispers of the personal messaging and e-mails that you do not see.
The problem is as they pay more this will get worse. This was the cause of the break between Oxford and Cambridge. At Oxford the students bought tutors and had the control and so the tutors said forget that we are going to set up a place where we teach as we want and students learn according to our standards. 

Student Feedback

When I took over as director of an MSc programme I had to review the module feedback of my colleagues. One of the lecturers consistently had amazing feedback. I found this odd as I saw his module content and from a teaching perspective it was amongst the worst I have seen. It was minimum activity and a huge amount of facts, long meandering lectures that showed no direction, a total lack of coherence and no sort of learning objectives. His assessments were weak and easily subject to plagiarism and the marks were so high that this brought comment from the external examiner who wanted to know why, and in fact we nearly had to lower them.
So why did he get good feedback? The students were not learning and certainly not learning deeply. It was fun and crazy and something they did not follow at all. But he gave them a nice easy assessment so that didn't matter. So they all gave him the great feedback.
Is the perfect situation where we give the students the answers to the assessment in the course, where we do not try to teach them anything. So we get a 100% pass rate and perfect feedback and they go out thinking they know it all.
There is something wrong with this. Students do not have to like you, but you do want to see some results. You do want to see learning. There is nothing worse than seeing a student panic and not respond to your teaching. None of us go out of our way to make students suffer. But have we allowed our teaching to be dominated too much by student feedback? When is is constructive and when is it an example of student apathy and laziness?

Monday, 19 March 2012

The successful online student

 Am I a successful online learner?
Sometimes. Today I am not as I am not really focussed when I am writing this. Last week I was not as good as I have been tired because of having to manage illness at home and also dealing with some challenges in setting an online assessment for my students. 
Sharing experience - I probably share too much - say too much, which is odd, I am in some ways an extroverted introvert. Some would say I love the sound of my own voice except I am very self-conscious. It took me years to cope with the stress of lectures and get into performing. Online there is less need for performance. In twitter I am more the watcher and less the exhibitionist, I follow more people than follow me. So I do not think it is easy to say that people share consistently. We are all a bit compartmentalised about what we share and when and with whom and in what environment. Online makes you more likely to share some things and less likely/able to share others.
Communication through writing - I hate formal writing. I always prefer conversational, almost spoken writing. So I do not think that there is a huge loss between writing and speaking. In Skype it feels like a conversation except we can all type at once, whereas we cannot all speak at once or nobody would understand anything.
Self motivation and self-discipline, time management - These are the most important factors. Motivation is not really a problem on this course as I am passionate about e-learning, or any way we can improve learning and make it more widely accessible. Self-discipline is a problem. With online courses as a tutor I have seen them rush off with a great excitement and interest before they lose momentum and drift towards assessment. I am the same you have to be more of a consistent plodder that a flash in the pan to be very successful. It is better to be Eeyore than Tigger. Moving from 12 hours to 1 hour then back to 6 hours then doing none is less successful than being consistent at four hours as if you give the course uneven time you get uneven results.
Accepting criticism - We are academics we never accept criticism. All the referees and grant reviewers in the world are idiots. They never have the big world view we have. Ignorance is wide-spread amongst those who are critical of our views. This is because we are in the field of reputation, we sell our reputations and anything that undermines our credibility is a threat. I am exaggerating but regardless of needing a thick skin to cope with reviewers we all have some soft spots.
Thinking it through - I showed above that I don't as I read criticism instead of critical thinking. So I am not always good. I know that I am biased about some things but we all have personal beliefs. What kind of constructivist, behaviourist, cognitivist are you?
Feel that e-learning can work - I have seen examples of the saboteur both online and face-to-face. I said in another post that I did team building with someone who did not believe in the approach we were taking to team building on that course. In that case it was a feminist sociologist who had an axe to grind. She was an expert in the feminist view of science and she made me want to burn The Female Eunuch much as women burnt their bra's in the 60s. Anyway after some cooling down time I can talk about it now. For me this is the most frustrating participant on a course. They are the most likely people to get me to say something inappropriate and Anglo-Saxon. Quite simply I do not want them there as they spread destruction through the course.

Thursday, 15 March 2012

Students and examples

I have been teaching students statistics for the last two semesters and one consistent feedback is they want me to go through examples. I have looked at each of my lectures and most of them involve half the slides going through worked examples and putting in real numbers. So I have been puzzled about this - You don't do examples.
Then it dawned on me, reading that students feel more comfortable when they see hand-writing compared to type. That and something the module leader had said about them wanting a chalk and talk, where I would do the examples.
The problem is they do not engage with examples where the whole solution appears, even if it appears in many steps over many slides. They actually need to see the writing process so that they can see it being worked out. It is not enough to show the calculations you have to show calculating!

Sunday, 11 March 2012

Managing Disaster

Things will not always go the way you expect. There were big differences between the three group discussions in terms of their activity and this sort of variability is a major difficulty when designing a course. You have to build in the flexibility and adaptability to cope with this sort of eventuality. What is most important is not losing student motivation, or making students feel left-out. When they lose the desire to participate then you have lost the battle.
There are a few possible solutions I was thinking about if you find a group is not participating as expected.
  1. Dummy students: create accounts for fake students that can be used to make sure there are active discussions on the course. These dummy accounts can be used to seed discussion groups and to make sure that important questions are asked.
  2. Give students very clearly defined roles: One of the weaknesses of online learning can be giving students too much freedom (Clark and Mayer). Students can be not very discerning or selective and they can be over-whelmed by too much choice. Three options is the most before people start to get confused (Gigeranzer).
  3. Make sure that interactions are assessed: If you are going to have a task it has to be linked to assessment to make sure that students will participate.

The 80:20 Law

I was looking at the blog usage on an online course I am doing and it will be interesting to see how the final blogs are but there seems to be an 80:20 law. 
This is not a bad thing. It just means that some participants are prolific posters. (This might be because they are either highly reflective or because they love the sound of their own typing ;-) ). Then there are others who are much rarer contributors. The ratio is that 80% of the posts will be contributed by 20% of the participants. These sorts of ratio are common in social networks and appear in many studies of social sciences.
The problem is if these indicate a pathology of the learning group. Do they mean that some participants feel left out? As was discussed in the second Skype chat, does this mean that some feel that they are imposters? Whatever the reasons for variation in engagement, it is important that these are not allowed to have an impact on learning.
What is the right way to address these issues?
  • Should some participants be excluded?
  • How can the tutor manage the interactions more successfully?
  • How can we best get a feel of inclusion?
  • Does watching matter?

Wednesday, 7 March 2012

Written voice

Feenberg raised the problems about the move from spoken to written when you use e-learning and that you lose the non-verbal aspects of a discussion. In some ways this can be dealt with by the written style you adopt. Reading the Skype transcripts they are very informal and there is not a real focus on the discussion topic in a rigorous way. They are conversational. They are very like the sort of chats you used to get in Yahoo rooms and Groups when they existed. You have to get used to multi-threaded conversations and everyone talking at once but after a while it becomes natural.
Rowntree talks about people chaging their language for online learning and becoming much more conversational and how it can take a while to adapt from academic language. This is something from the debate. The proposals and oppositions are written like you would for a research paper - with evidence and references. In a spoken discussion you would not be as exact, it would have a completely different feel to it. It would be much less logical and much more rhetorical.

Academics as strategic learners

To be an academic you have to have done well as a student. But is that because you were a deep-learning conscientious student or because you were a strategic shallow learner. I was doing a teaching course and one of the other participants confessed that she had always been a strategic shallow learner. She would just cram the week before the exam and then forget everything as soon as the exams were over. In the end she left lecturing and decided to take a gap-year, as she had gone from student to post-grad to lecturer.
Thinking about the tasks we have been assigned, are people on the course acting as strategic learners? We are all busy and trying to balance work, life and studying so is it reasonable to take a strategic approach? Not everyone have completed the wiki task and the discussions have not been very active. Not linking tasks and assessment is definitely a bad idea if you want to make students participate.

Tuesday, 6 March 2012

Gender and Computers

In Devon they often give gender to objects and ships are always considered women. A lot of European languages still have genders and so objects have gender. Feenberg talks about the problems of text and isolation and the lack of relationship in CMC but does the gender you associate with technology make a difference to how friendly you find using that technology?
A friend was playing online Scrabble - they are German and she thinks of the computer as a man that she wants to show that she is cleverer than. This got me to thinking about the gender I give computers. I have always felt that my computer was female. I always want to get it to cooperate and to calculate the right solution for me. I always want my programs to work without errors. If I gave my computer a voice like those in science-fiction it would definitely be female. A calm voice, firm but not motherly a youthful independent voice.
It was just an odd thought but we all create voices in our head for thinking.

Monday, 5 March 2012

Biggs and the SOLO Taxonomy

I use Biggs' solo taxonomy for as many marking criteria as I can get away with. I find that writing a detailing marking scheme can never fairly reflect student performance. When you write it you always have in mind the way a student is going to answer the question but you never manage to cover the full diversity of responses. I found that when I have been marking the statistics assignments I wanted to change the marking scheme as some students had done well but not managed to address the specific points of the criteria. This is always a problem when you have large classes and the marking has to be shared out. You want to make the marking as transparent and objective as possible but in the end this often leads to less fair marks than a more generic and subjective set of criteria.
That is why I like Biggs as I align it to the grade boundaries:
  1. Pre-structural - Level 4 (fail pass at level 6).
  2. Unistructural - Level 5 (3rd at level 6)
  3. Multi-structural -  Level 6 2:2
  4. Relational - Level 6 2:1
  5. Extended abstract - 1st - MSc Distinction.
Then you can fill in the evidence as you read the work and you are not thinking there is 1 mark for this and 2 for that ...

Saturday, 3 March 2012

Synchronous vs asynchronous

If you look at the activity in synchronous and asynchronous tasks the synchronous tasks win hands down! This might only be for this particular group as I have seen asynchronous discussions work well, but these are often tutor directed. 
I expect that people will contribute much more to the Skype sessions than to the discussion board groups and this suggests that the discussion board is not an effective technology when used in this way.
There is too much expectation on the "students" to take control of their learning and this just does not work. Even the most active groups have only 50% of the group participating. Not having the discussion linked to assessment makes student apathy to the task even more likely.
While you can have lurkers in synchronous chats and this is not the end of the world, it becomes much more serious in asynchronous discussions as you end up with the sound of one hand clapping.

Thursday, 1 March 2012

Picking the wrong technology

Picking technology you need to also look at where the future of web-technology is going to go, as well as the frameworks in which you are going to need that technology. Accessibility is a serious issue if you pick the wrong technology. I have developed some materials in Flash in the past but this is not accessible to some students with disability. Image-maps were often used early in the creation of web-pages but this is not a disabled friendly technology and so now it is seen less often.
In general the use of plugins such as Flash or Java can exclude possible users for a variety of reasons;
  1. Disability access - some users use text only browsers and so visual only navigation and content cannot be used.
  2. Security access - some users are behind firewalls or do not have administrative access to install plugins.
  3. Bandwidth access - a lot of plugin based technologies need large bandwidth (Java based applets can be very large).
  4. Hardware access - not all students have video or microphone and if they do this increases bandwidth requirements.

The world wide web consortium has tried to address these issues by creating the new HTML5 standard in which more of the media aspects of the web are actually built into the browser rather than requiring plugins. This presents an opportunity to develop courses in a more customisable and interactive way, even without a VLE, but the cost is going to be a need for much larger development times and a much more technically involved set of authors and author teams.

Can groups self organise?

My gut feeling is no!
I have been watching the discussion boards for our second task in my online learning course with interest, to see how the groups go about it. There seems to be an active member who poses the question and gets the balls rolling so that the participants assign the different roles
My question is whether this person feels themselves or if the other feel them as a leader. Do people elect a leader or are they self-elected or are the groups leaderless?
This was a serious issue when I did a team building course, and in the end we had to take a 2 hour calming down session to stop everyone hitting one another.
So does the group ever self-organise or is there someone pushing it along?