p-value cannot determine importance of results

this is refreshing news via nature.com in the morning. the American Statistical Association (ASA) warns that the P value “CANNOT determine whether a hypothesis is true or whether results are important.” (emphasis added).

a 6-principle list that addresses misconceptions and misuse of the p-value, can be found in the ASA statement:

  1. P-values can indicate how incompatible the data are with a specified statistical model.
  2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
  3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
  4. Proper inference requires full reporting and transparency.
  5. A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.
  6. By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.

the press release statement goes here:
Screenshot 2016-03-08 07.47.15

good to take note for those of us into educational research, and have the tendency to rely on quanti methods. and its our duty to correct the misconceptions of non-researchers on the use of p-value too.

facilitator’s log (12541 primary)

last friday, we held our first session of ICT for SDL and CoL Clinic for primary school teachers (TRAISI code: 12541, or 12541-00003). as the traisi code suggested, this is the 3rd round of primary clinic. the first round was facilitated by dear wu jing cher, and the second round was facilitated by me when wj left for her full-time studies.

this 3rd round of primary clinic returns me to a dual-facilitator role – to facilitate the learning of both teachers who signed up for the course, and the instructor of the course (Wen Yun who joined us to take up the position vacated by WJ). unlike the previous round of handholding, WY is expected to conduct this course without a chance to ‘witness’ nor much time to co-construct the (ideal) conceptions of the course.

as a reminder to self, this blended learning course is designed in attempt to work towards 2 ideals: 1. to find a new approach towards CL teachers’ PD (tying short-term courses, with a larger CoP in the long run); 2. to be a student-centered course situated within a social constructivist paradigm, where learning is facilitated through discourse (both f2f & online).

to facilitate our further conversation, in this log, i would like to capture some observations and thoughts. the ‘source’ of these ideas came from our conversations prior and after the f2f session, and observations during the f2f session).

  • “我是传统的师范毕业生。”
  • – this line came up i think at least 2 or 3 times during our conversations. i am not sure if WY realised she had emphasised this idea many times, but my guess is this is a reflection of an internal struggle with theoretical understanding, and existing practices in teaching. “传统” & “师范” connote a teacher-centered, teacher talk-dominated, learning = passive copying and listening type of teaching. the source of internal struggle probably came from (1) her deep understanding of CSCL principles based on her phd studies, (2) the ideal #2 of this course that demand (1) to be put into practice. this is perhaps a reflection of the saying ‘expoused beliefs is one thing; actual actions is another’.

  • “my beliefs about learning have changed, but i may have difficulty putting it into practice” (approximation, can’t recall exact words)
  • – yes, what i thought of when i heard this is “if action is powered by beliefs, actions thus reflect our beliefs”. if we are unable to do certain thing, is it that our beliefs have not changed to the extent we thought it would have changed? change in beliefs over time will be something useful to track (researcher’s talk) 😛

  • “我从来没有感觉上课那么没底儿。”
  • – what does “底” mean? my guess is fixed structure coupled with fixed load of contents to talk about and pushed into the black boxes of the participants? clinic has a course structure, so structure is not the issue. so we are left with the contents part. following the student-cantered ideal, we will be facilitating discussions and learning in ideas that participants would raise in the course of our interactions. this is really the most unsettling feeling that i get based on my personal experience. some qns that have ever passed through my mind include,

    “what if there’s things that i cannot answer?”;
    “what if participants think badly about this ‘teacher’ who doesn’t appear to know anything becos s/he doesn’t spoon-feed us the textbook defns, and provide us with loads of contents and resources?”;
    “what if there are important ‘learning’ to be taken away but the idea doesn’t surface from the participants at all?”

    for the 1st qn, i convinced myself that no one can truly know everything, but it’s important that we are willing to go and find out things that we don’t know, and know how to go abt finding out what that we needs to find out. and this is an important lesson about ‘learning’ that i want my participants to takeaway, if they takeaway; and i do this by modelling the behaviours (it’s OK to say i don’t know; i demonstrate how i go about finding out abt new things; i demonstrate that learning takes TIME, a forgotten or simply ignored fact in the production mode that we conduct teaching in schools (too little time, too many things to be taught and assessed. 填鸭填鸭再填鸭; stuff, stuff, stuff)

    for the 2nd qn, this is the greatest ‘risk’ that we have to take, especially if participants possess naive personal epistemology, and deeply entrenched in a content-cum-resource loaded expectation of ‘learning’. Victor mentioned he had his fair share of such experience, cos we all need to face the ‘reality’ of SFT. based on my experience so far, i just have to hold on to my beliefs and ideals, and trust that teachers will be able to see and appreciate it at the end of the day. if there are some starfish that i couldn’t move this time, it’s just not enough time i have to engage them. learning takes time; changing of beliefs take even more time i believe (and i think we can find literature to support this, which i haven’t really done so, yet.)

    for the 3rd qn, over the past 3 secondary clinic and 1 primary clinic that i have facilitated, i am not so worried about this anymore. cos if the issue is so important (for e.g., conception of ‘affordance’, TPACK), we will be able to make links to it one way or another. probably that’s why as teachers, we always say teaching is a science, but is also an art. weaving is the ‘art’ part.

below is a record of things that i caught my attention (incl. being shown, being said):

– defining what is ‘e-learning’, and relate it to MOOC
– “ICT=vitamin是相对主流的用法。”
– “这些原则(ref to the 4 interaction guidelines)你的学生也可以用。”
– “我们可以聚焦一些课题来讨论。” (this is not suggested by WY)

a few other notes:
– too little time is left to go through the details of the activities, and check for understanding that participants know what to do where to do when to do what
– there’s a rich repertoire of ideas surfaced by participants that i would use to build on. use them to raise questions to keep them thinking, no answers’ required there and then.

that’s all for now (:

which level are you at?

no, this is not a RPG question, nor a rhetoric qn.

was reading and saw reference to John Biggs (1999)’s Levels of Teaching Competence in his other article ‘what the student does? teaching for enhanced learning‘.

so, using RPG-like language, i can probably ask a teacher “which teaching level are you at?”, or “which teacher level are you?” 😛

a quick list of the focus of the 3 levels of teaching competence:

level 1: focus on what the student IS
level 2: focus on what the teacher DOES
level 3: focus on what the STUDENT does

level 1’s focus is on knowledge transmission. teacher’s responsibility is to lecture, and assume students will ‘absorb’. if students do not learn, ‘blame the student’ (in terms of deficit in ability, attitude, study skills, motivation (Samuelowicz, 1987).

level 2’s focus is very much on what the teacher does to transmit knowledge (concepts and understanding) to students. a teacher will aim to work at equipping oneself with ‘an armoury of teaching skills’. PD is focused on ‘HOWTOs’. the deficit now lies with the teacher. ‘blame the teacher’ for being incompetent (teaching is a bag of competencies) if s/he doesnt transmit knowledge well.

level 3’s focus is on students learning. one may argue that this requires level 2’s competencies as a basis. perhaps so. but the focus is on what the student does, and the key qns for consideration is “what it means to understand those concepts and principles in the way we want them to be understood?”, and “what kind of teaching/learning activities are required to reach those kinds of understanding?” (p.63)

level 3 is ‘student-centered’ teaching. 1 & 2 are teacher-centered.

writing, marking

saw this article via a fb post/link this morning:

2016-02-17_082902Responding to Student Writing — and Writers

and i thought what the author pointed out are indeed in line with the spirit of social constructivism (meaning-making occurs in discourse/dialogue), and assessment for learning. 6 ideas to get it write:

  1. Differentiate comments on drafts from those on final essays.
  2. Give grammar lessons their own time and space.
  3. Create a partnership with students across the drafts.
  4. Extend these writing-based partnerships by having a class-wide conversation about commenting.
  5. Establish a class language for comments.
  6. Be encouraging.

the ideas assumed that composition writing is a process that involves drafts before finals. process writing is in the true spirit of developing writing competencies in our students. i must confess that i did not do this in the past. but if i were to be back in the classroom, this would be a must-do. otherwise, how can i expect my students to improve in their writings when (1) a one-off marked and return exercise may have too many issues to tackle (字、词、句、段、篇 all 5 areas add up is A LOT A LOT A LOT), not to mention this turns every essay into (2) a summative assessment in disguise.

development takes time for the learners. and in this case, it would most likely take up more time of the teachers. one thing that must definitely be looked into: the number of essays to be written as stated in the SOW. it would probably make a good experimental study to compare a class of students who completes only 4 (let’s say) ‘formative’ essays vs. a class of students who completes the usual 8 (let’s say) ‘summative’ essays.