Orwell Award Announcement SusanOhanian.Org Home


A Math Purchase You Should Care About

Publication Date: 2004-11-18

I wish the time and money were available to uncover every article written about education. Here we have a seemingly simple announcement of a district purchasing a math program. A very cursory look at a few details shows that it isn't simple at all. I am not an investigative reporter and I don't pretend to have more than scratched the surface. But I hope that what I did reveal raises questions. For starters, I hope people ask questions about pedagogy, about professionalism, and about profiteering. Those of us who care about education must ask these questions of every item we read.


The wonders of technology: Ohanian comments are in red; the news article that provoked this rant is in black, as is the U. S. Department of Ed report; statements by I CAN Learn are in blue, as are words of praise. This newspaper article provokes questions--about the content of the math program offered and about the company providing it. A bland report contains a few positive words such as self-paced and motivation. Of course self-paced is one of those words that deserve a footnote, with one person's self-paced being another's skill drill direct instruction. In fairness, this is the report of a school board meeting. We can hope the investigative journalism will follow.

In any case, I provide some information about the company and its delivery system following the article. Some of the info has profound implications for teachers who still regard themselves as professionals.


45 New School Math Labs Approved
Aesha Rasheed
Times-Picayune
November 16, 2004
http://tinyurl.com/4s6ck

Dramatically expanding a computer-based math program, the Orleans Parish School Board voted Monday to double the number of math labs at a cost of $6 million.

The expansion, a proposal by Superintendent Tony Amato, will put 45 new math labs at 38 public schools, bringing the district's total number of computer-based math classrooms to 99.

District officials credit the self-paced learning labs with boosting math scores on the most recent LEAP test.

"Children assigned to I Can Learn classrooms are much more motivated," said Mary Thompson, administrator of the district's math and reading programs.

Board President Cheryl Mills complained that the agenda item was confusing and rushed. "All of this unclarity means we're operating by the seat of our pants," she said.

Included in the administration's request for the labs was a note from Amato that he has asked the state Board of Ethics to rule on whether a contract with JRL Enterprises, the company that sells the labs, would be unethical since the company has hired school district teachers as consultants. The district has received verbal approval from the ethics board, and is awaiting a written opinion.

Mills ultimately voted in favor of the contract, as did five other board members. Board member Gail Glapion was absent.

In other business, the board learned that finance department officials have investigated 123 of the 861 people suspected of collecting erroneous or illegal paychecks, and found most of the payments were legitimate. Of the more than $700,000 in questioned payments to the 123 people investigated, about $262,000 turned out to be incorrect, Area Superintendent Matt George told board members.

George said the school system has begun contacting people who were incorrectly paid and asking them to return the money. He could not say how much has been returned.

Board members said the identities of the cleared people should get the same public attention that they received when they were listed in The Times-Picayune among the names of more than 2,000 current or former employees whose pay was in question.

"A lot of people were really upset by this matter," Mills said. Publishing the list was "an irresponsible action and we are as anxious to clear the names as they are."
. . . . . . .
Aesha Rasheed can be reached at arasheed@timespicayune.com or (504) 826-3378.

I can Learn website is long on PR piffle and short on pedagogy.

http://www.icanlearn.com

Don't you love outfits that promise to deliver a complete education system, software or otherwise? When was the last time anything was complete in your life? When someone says their product delivers everything you need, you know they're lying. In this case, as we will see below, everything you need is a code word for not needing a certified teacher to run the lab

And you'll see below what recognized by the U. S. Department of Education means.



The program is a complete education software system that has been recognized by the US Department of Education. It began by teaching Algebra to 200 students in an inner-city school in New Orleans during the 1994-95 school year. Today the I CAN Learn program has expanded to cover 15 states and over 50,000 students. Its range is also expanding to include school grades K-12 and Colleges.

http://tinyurl.com/6845f


What do you think existing workforce means? If it were teachers, wouldn't they just say teachers?


I CAN Learn Education Systems is a comprehensive mastery based math program for all learners. It is especially effective for use in training the existing workforce for the needs of the future.

http://tinyurl.com/4zo6z


Here, I CAN Learn inserts credentialed teachers into their promo material:

Using the latest technology, interactive lessons, real world applications, and individual tutoring from credentialed teachers, the I CAN Learn Education System teaches Pre-Algebra and Algebra to students in a self paced learning environment with a 1:1 student to computer ratio. Tutors monitor student activity and use the I CAN Learn classroom management system to identify struggling students who are having trouble mastering coursework enabling them to work individually with those students. The I CAN Learn Education System is especially effective with at risk students and has been designated a ?Promising Mathematics Program? by the US Department of Education. According to the USDE, the program?s teaching and assessment methods are sound, learning goals are comprehensive, and instruction is designed to improve academic performance for a wide spectrum of students.


Credentialed Teachers. Hmmm. How does this claim square with the ad below? A high school graduate will work in lab with and without an instructor present.

Lab Assistant
(?I Can Learn? Math Lab)

Essential Job Duties: Work in lab with and without an instructor present in order to tutor/assist students with the mathematics lesson; Work evenings and Saturdays.

Qualifications: High school diploma required; Associate?s degree preferred. Strong mathematics skills; Good communication and interpersonal skills; Proficient in use of computers.
Review of applications will begin immediately and continue until filled.

To request an application, contact the Personnel Office at Phillips Community College UA, P.O. Box 785, Helena, AR 72342, or call (870) 338-6474, ext. 1271 or Email ejames@pccua.edu . AA/EOE.
http://tinyurl.com/3mtjt


When offered the opportunity to add keywords for a searchable database, sales people know what to do. It is worthwhile to study this list. How many are weasel words? How many are code words for something else?

Individual tutoring, Small group interaction, Mathematics, Algebra, Pre-Algebra, Math basics, Math Resources, Learning Center, Education Center, Tutoring, Tutoring Services, Integrated Math, Assessment, Skills Assessment, Educate, Education, Online Resources, Education Programs, Teachers, Certified Teachers, Personalized Programs, Standardized Tests, Learning Objectives, Learning Goals, Motivational Program, Learning Resources, Motivation, Educational Software, Computer Assisted Instruction, Accountability, Quantitative Metrics, Daily Assessment, Proven Effective for low income, at risk students using Scientifically Based Research, Recognized by the US Department of Education as a ?Promising Mathematics Program?. http://tinyurl.com/3k5aw


Don't miss who praises the product:

?The patented I CAN Learn? system delivers a complete course curriculum while tracking the progress of each student. It also handles such mundane tasks as taking attendance assigning homework and administering and grading tests.?
- Forbes Magazine



I find it interesting that nowhere on the I CAN Learn site could I find any talk about pedagogy. Nowhere did I find an evaluation or a blurb from a math expert. Nowhere could I find who wrote the material presented to students. The product is listed on the government's What Works Clearinghouse site, and you'll find the report below the teacher's letter.

This letter from a New Orleans teacher to a member of the school board (herself a former teacher and administrator) is on the I CAN Learn website, along with a copyright notice.



Results - What They're Saying, Letter #1

FANNIE C. WILLIAMS
MIDDLE SCHOOL
11755 Dwyer Rd. New Orleans, LA 70128
June 21, 2001
Ms. Ellenese Brooks-Simms
New Orleans Public Schools
3510 General Degaulle Dr.
New Orleans, La. 70114

Dear Ms. Brooks-Simms:
As a math educator and a lifelong citizen of New Orleans, I can?t help but be concerned about our children at those lowest performing schools. When I first came to the system from the archdiocese, my first teaching assignment was at Live Oak Middle. I know first hand the hard work and dedication exhibited by those teachers. Teaching in a public school for the first time was an eye-opener because the students came with a lot of baggage. Over the years I have taught math from 3rd grade to college, in parochial, private and public schools and have learned that the only true difference between the children in our best schools and those in our worst is how we treat them.

For the past two years I have had the privilege of working at Fannie C. Williams in an I CAN Learn math lab. It has made a tremendous difference in not only math scores, but in the way that my students view math. Last year 70% of my students passed the math portion of the LEAP. Another 20% were within a few points of passing and after summer school and retesting were able to go on to 9th grade. This year 92% passed, with another 7% able to go to 9th grade after summer school. That means that 99% of my students will advance to 9th grade.

More importantly the attitudes of these students toward math completely changed. At the beginning of the school year, students told me ?I can?t do math. I?m not good in math. I?ve never been good in math, so I know I?m not going to pass LEAP.? One of my students from the previous year visited at that time and heard the students? apprehension about the test. She told them, ?Oh, don?t worry, you have Mrs. Austin and the lab. You?re going to pass.? Later, after experiencing success in I CAN Learn, students became more confident. By the time they took the test, they were even cocky. ?Mrs. Austin, that test was easy! I know I passed!? Even though I had evidence of their success in the I CAN Learn program, I was still skeptical as to whether they would do well. On the LEAP there are many factors that influence the result, including reading errors, erasures, bubbling errors, etc. But when the results were published, I found that the children were, indeed correct. They had a right to be cocky. They had a right to be proud.

Unfortunately, the results for the rest of the school were not as good. Only 32% of the students in non-I CAN Learn classes passed the math portion of the LEAP. Their students were apprehensive going into the test and not very confident after the test.

I have spoken to colleagues across the district, and I have heard that I CAN Learn students at Booker T. Washington and Woodson also did well and out-scored their non-lab counterparts. The lab is not a substitute for a good teacher I by any means. But pairing a good teacher with the benefits of the individualized instruction offered in the lab is the best way to build math literacy in a short period of time.

Children in our best schools are used to success. They expect to master what they are taught. Children in our worst schools are used to failure. They don't expect to master anything. The I CAN Learn program shows them that they can experience success. They are given immediate feedback on quizzes and whether the student masters a lesson on the first try or the 15th, a perfect score is rewarded with rockets going off to celebrate his victory .The benefit of success in math often carries over to other subjects. By the time the third quarter came around, an overwhelmingly large number of my students were on the honor roll, some for the first time since kindergarten. As many told me, once they were able to pass math, the hardest subject, they knew they could pass the rest. They then began to work harder in everything else.

My heart breaks for the students and teachers at our lowest performing schools.

How awful to be labeled "the worst" in the district! Children with already low self-esteem are being told by the media that everyone else is better than they are. This cannot continue. Please consider adding I CAN Learn labs to these schools and all of our middle schools. I know that the cost is high but the price of continued failure for our students is even higher.

I will gladly assist in any way that I can. I will help train teachers, work on correlating activities, develop daily lesson plans, and anything else that will assist teachers and students to make the transition from failure to success.
Sincerely,

Joiclyn Lloyd Austin
? 2002 JRL Enterprises, Inc. - I CAN Learn? Education Systems is a Registered Trademark of JRL Enterprises, Inc.
http://tinyurl.com/64r2v


Here is how the product is listed on the U. S. Department of Education site What Works Clearinghouse. A brief description is accompanied by this lengthy report that obfuscates as much as it reveals. I know we're supposed to care about the distinction between quasi-experimental studies and randomized controlled trials, but it's a hard sell. Disturbing to me is the fact that when you click on the studies, you get a message in red Coming. What is provided are U. S. Department of Education summaries. Well, we trust these guys, don't we? This becomes an issue when you realize that the four studies critical of the I CAN Learn? Mathematics Curriculum do not meet the federal government's evidence screens. Big Brother decides what works and what doesn't work.

The I CAN Learn? Mathematics Curriculum is an interactive software system for pre-algebra and algebra. It consists of 326 lessons from basic mathematics to advanced algebra concepts. Designed for students to learn at their own pace, the I CAN Learn? program is intended primarily for use in grades 7 through 10 by ethnically diverse, inner-city students.
--------------------------------------------------------------------------------

For Middle school students.
--------------------------------------------------------------------------------

Findings The one small randomized controlled trial (254 students in one school) found that I CAN Learn? students scored significantly higher on the state math test than students in the comparison group who used a ?traditional? curriculum. All three quasi-experimental design studies found in at least some of the analyses that I CAN Learn? students outscored comparison students. One study found that some groups of I CAN Learn? students scored significantly higher than comparison students but that other groups scored significantly lower than comparison students. The second study found that I CAN Learn? students scored higher than comparison students, but the differences were not statistically significant. The third study found that I CAN Learn? students scored higher than comparison students, but a flaw in the analysis makes it impossible to accurately determine the significance of the finding.
--------------------------------------------------------------------------------

Evidence base 1 randomized controlled trials meet evidence standards.
3 quasi-experimental design studies meet evidence standards with reservations.
4 studies do not meet evidence screens.

--------------------------------------------------------------------------------

Evidence limits The evidence base for the I CAN Learn? Mathematics Curriculum includes four studies, only one of which was a randomized controlled trial. The others were quasi-experimental design studies. Quasi-experimental design studies provide weaker evidence of effects because unmeasured differences between the groups can affect the findings. None of the studies provided evidence of strong implementation. Three studies used small samples (fewer than 350 students), and the larger study had flaws in analysis that could bias the findings.
--------------------------------------------------------------------------------

Scope of use The curriculum was first implemented in 1995. By October 2004 it was being used in 23 states by 501 schools. The software has typically been used in Title I schools and by at-risk and minority children in large urban school districts.
--------------------------------------------------------------------------------

Developer and contact JRL Enterprises, Inc., 400 Poydras Street, Suite 100, New Orleans, LA 70130; telephone: (504) 263-1380; www.icanlearn.com.
--------------------------------------------------------------------------------

Profile

The I CAN Learn? Mathematics Curriculum is designed to help ethnically diverse, inner-city students in grades 7 through 10 achieve parity in higher level mathematics and problem-solving skills. It uses an interactive software program to teach pre-algebra and algebra. The I CAN Learn? Algebra course, which was used in all five of the studies included in this intervention report, consists of 177 algebra lessons. The developer describes the curriculum as meeting National Council of Teachers of Mathematics standards and configurable to meet state and local grade-level expectations.

Lessons are grouped much like those in a textbook chapter. Students receive verbal and visual assistance in progressing through the lesson. After completing a lesson, students take a cumulative review of the concepts taught, and teachers can monitor student progress through real-time assessment. The software also handles taking attendance, assigning homework, and administering and grading tests. Tests are randomly generated from a test database, so no two students receive the same test. Similarly, homework is generated for students, and the software conducts random homework checks to make sure the assignments are being completed.

Implementation of the I CAN Learn? Mathematics Curriculum involves the creation of an I CAN Learn? classroom with computers, custom desks, software installation, and typically three years of support from the developer. Desks are set up in a ?conference room? arrangement, which the developer argues encourages cooperative learning and group work.

Teaching

Though much of the instruction is individualized for students in the I CAN Learn? Mathematics Curriculum, teachers are expected to help determine the content of the lesson and other aspects of the class. At the beginning of the year, the teacher determines the homework assignments, lesson organization, lesson presentation, manipulatives, assessments, and grade evaluations. The program follows the teacher?s lesson plan and provides constant feedback to both the student and the teacher.

As part of the I CAN Learn? system, teachers receive an initial two days of training in how to use the system. Teachers also have ?best practices? training and unlimited access to training and on-site technical and pedagogical support in the classroom for three years. The I CAN Learn? personnel call or visit I CAN Learn? teachers weekly.

Typical lesson

Each I CAN Learn? lesson follows a six-part format: pretest, review, lesson presentation, guided practice, post-lesson quiz, and cumulative review. The pretest covers material from the upcoming lesson. If students miss one question on the pretest, they continue into the current lesson. If students get all the questions right on the pretest, they may advance to the next lesson if the teacher enables the software?s advancement option.

In the review section, students are presented with prerequisite material that is necessary for understanding the current lesson. During the lesson, a ?cyber-teacher? presents concepts and problems that increase in complexity as the lesson progresses. Students work out the problems presented in a notebook and compare their answers with the ones presented by the program. If students complete the problems incorrectly, the ?cyber-teacher? is available to work through the problems in detail?to help students figure out where they may have made an error.

The post-lesson quiz contains a variety of problem styles that the developer indicates conform to national standardized tests. The classroom teacher determines completion time, the number of questions, and their respective difficulty.

After students complete a group of lessons, they go to a cumulative review, which presents three problems from each lesson. If more than one is incorrect, the lesson presentations for those areas are reviewed.

Scope of use

By October 2004 the I CAN Learn? Mathematics Curriculum was being used in 23 states by 501 schools. It is typically used in large urban school districts and smaller rural school districts whose students are predominantly at risk and members of minority groups.

Cost

The cost of an I CAN Learn? classroom depends on its configuration and terms of support. A typical full installation?30 workstations in a classroom with all curriculum and classroom management software, computer hardware, network wiring, furniture, and three years of comprehensive onsite educational and technical support?costs $300,000, a one-time expense.

Study findings

Randomized controlled trial

One small randomized controlled trial with 254 students in one school (Kirby 2004) found that I CAN Learn? students scored significantly higher on the state math test than students in the comparison group, which was using a traditional math curriculum.

Quasi-experimental design studies

One quasi-experimental design study (Brooks 1999) found that 7th- through 10th-grade students scored higher on a researcher-developed test than students in the comparison group; however, due to limitations in the way the analysis was conducted, it is not possible to determine whether the difference is due to the curriculum or to chance. A second quasi-experimental design study (Kerstyn 2002) found that 8th-grade I CAN Learn? students scored significantly higher on the Florida Comprehensive Assessment Test (FCAT) than comparison students for two classes (MJ-3 pre-algebra classes) but scored nonsignificantly lower for the remaining four classes (Algebra 1 honors, Algebra 1, MJ-3 advanced, and MJ-3 pre-algebra). A third quasi-experimental design study (Kerstyn 2001) found that 8th-grade I CAN Learn? students outscored comparison students on the FCAT for all five classes, but the differences were not statistically significant.

Strength of the evidence base


The WWC collected more than 800 studies for the Middle School Math Curriculum review. Eight were on the I CAN Learn? Mathematics Curriculum. Of these, one met standards while three studies met standards with reservations. The other four studies did not meet standards. They were quasi-experimental design studies that did not account for pre-existing differences between groups with matching or equating.

Studies were rated according to the strength of their causal evidence. Studies that placed students into the intervention and comparison groups randomly (randomized controlled trials) without notable design or implementation flaws are classified as meeting evidence standards (). Other studies that use comparison groups (quasi-experimental designs) and randomized control trials with notable flaws are classified as meeting evidence standards with reservations ().

Studies are further rated for intervention fidelity, outcome measures, breadth of evidence, reporting on subgroups, analysis, and statistical reporting. That information is provided in study reports, but does not affect the overall rating.

None of these studies provided evidence of strong implementation. Implementation was not discussed in two of the studies. And there were variations in implementation in the other two studies.

All four studies used valid tests that appear aligned to the curriculum, including state tests in three of the four studies. The fourth study demonstrated the reliability and validity of its locally developed test.

Although each study looked at a narrow population and settings, collectively they looked at and reported on 8th- through 10th-grade students of diverse racial and economic backgrounds in urban and rural districts. Three provided enough information to compute effects, but their samples were small. One other study had flaws in the analysis that may bias the findings.

http://tinyurl.com/59nv2


FAIR USE NOTICE
This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of education issues vital to a democracy. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information click here. If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner.