Media Search:



From clues to facts to convictions – UND Today – University of North Dakota

UND Forensic Science students tour North Dakota State Crime Lab, learning firsthand how evidence analysis works

Editors note:Due to a technical error when the UND Today newsletter was sent out on Tuesday, the lead photo for this story did not transmit properly. As a result, the story is being sent out to readers once again in the UND Today newsletter of Thursday, March 23.

Authors note: When the director of UNDs Forensic Science Program asked if I would like to tag along with her students for an all-day field trip to Bismarck, I jumped at the chance.

After all, the last fun field trip I could recall was when my sixth-grade class in Devils Lake, N.D., crammed into a sticky yellow bus and headed to the Big Top in the big city of Grand Forks. Our end goal, of course, was to see the Shrine Circus in the Hyslop Sports Center.

This time around, our destination would be the pristine North Dakota State Crime Lab where the only clowns would be the criminals, and the only talk of elephants would relate to an animal tranquilizer 100 times more powerful than fentanyl.

Warning: You may find parts of this story disturbing. If you dare, come along with us for the ride

* * *

We gathered before sunup myself, about three dozen students and Assistant Professor Lavinia Iancu in the Chester Fritz parking lot. Coffee in hand and wearing comfies and caps, some of us looked fresh out of bed yet ready for adventure.

We had a long haul ahead, but no worries. We had WiFi, so thered be plenty of time for Netflix and naps before wed need to crack open the three coolers stocked with sack lunches and sodas. (Thank you, UND Catering.) If we got hungry sooner, well then, our bus driver Kevin was sharing his butterscotch candies.

OK, listen up, Iancu said, lifting a small red-and-white megaphone to her mouth before lowering it and adding with a wry smile, I have a siren, and Im not afraid to use it.

We laughed up front, but she apparently meant it and showed us by quickly flipping the switch to release a high-pitched wavering scream all the way to the back of the bus. She had everyones attention now.

OK, so the plan for us today is to go visit the DNA, toxicology and forensic chemistry divisions. We will rotate as three groups and have discussions with the experts, she explained. They are going to do demonstrations for us. So, we will see how much interaction we can have. They are very busy and working on cases all the time, so we are lucky not everybody gets this kind of opening. This is your chance to ask questions.

The tour truly would be a one-of-a-kind learning experience. Not only would it mark the first time in 17 years a UND class had visited the lab, the students ranging from freshmen to seniors also would have all-afternoon access to about a dozen forensic experts.

This is a first, but I hope well be able to do it at least once a year, Iancu said. Wed also like to establish some summer internships. Unfortunately, thats not possible right now because theyre understaffed, but we are working on it.

I absolutely love what I do, but forensic science can be very stressful, hard work. For me, its also incredibly rewarding, but I want my students to understand that as forensic scientists, theyll have some very serious responsibilities. Theres nothing closer to reality than what were going to see today.

After all of the passenger names are called and checked off and a buddy system established for rest stops it doesnt take long for everyone to settle in. I figure its as good a time as any to meet my neighbors.

To my left was senior Yuliet Monatukwa, a double-major in Forensic Science and Chemistry. Already accepted into two graduate chemistry programs, Monatukwa said she hoped the tour would give her clarity on another possible career path one focused more closely on analytical chemistry and toxicology.

It so happened that the very next day, she was flying to Denver for an interview with the University of Colorados Toxicology Department.

Im really looking forward to finding out what it takes to be a criminal investigator at a crime lab, Monatukwa said. Dr. Iancu has given us so many hands-on experiences, and I think that really has cemented my decision to do something down that path. A person really has no idea about the application of theory until you actually try to apply it yourself. So Im curious, and Im going to ask a lot of questions today.

Malia Wellens, a senior and double-major in Forensic Science and Criminal Justice Studies, said she already had settled on graduate school at the University of Connecticut in New Haven.

Though she also started out at UND as a chemistry major, she said she switched to Forensic Science with a focus on forensic biology.

I think it fits more with what I want to do, she said. Once I learned about DNA analysis and the different branches of forensic biology, I was like Thats my bread and butter. I love it.

Just a month earlier, Wellens began working for Iancu as a junior research assistant studying the effects of Clonazepam and Flunitrazepam common drugs, known as benzodiazepines or psychoactive drugs, prescribed for seizures and panic disorders on blowfly development. (The results of that research eventually could alter how scientists estimate time of death, but thats another story.)

In the blue bucket seat next to Wellens was Jaden Eviota, a junior and another double-major in Forensic Science and Criminal Justice Studies.

He said: As much as I find the lab interesting, I still think Im more into getting out in the field to do crime scene investigation. Growing up, I found that pretty much everything I watched on TV or the movies was forensics-related, so thats what got me interested at first. Im not sure exactly where Ill land, but all of this is helping me shape that decision.

And in front of me sat Nicolette Ras, a Pre-Med senior and double-major in Forensic Science and Criminal Justice Studies. She recently began working part time for Iancu as a teachers assistant doing everything from setting up lab trays and equipment for classes to handling logistics for the days field trip.

Ras said her ultimate goal is to become a medical examiner, and thanks to Iancus tip about a job post, shes now learning the ropes as an autopsy technician at UNDs Department of Pathology. The office handles cases of suspicious or unattended deaths in North Dakota.

Wearing a quirky, sitcom-inspired Schrute Farms sweatshirt with a giant beet imprint, Ras added, Its all so fascinating. I just love forensic science.

On the TV shows, its the next day and you have a DNA result. Thats not quite how it works, but its still going to be pretty cool to see where all the evidence were packing up is going and to actually meet some of the people who are processing it. Im so excited for today.

This tagalong agrees. Theres no place an avid Ann Rule reader and true-crime junkie would rather be.

It takes a little more than four hours to reach our final destination, which turns out to be a gray building in a complex of other nondescript, one-story concrete buildings. First impression? From the outside, meh, nothing fancy. Inside, its state of the art.

UND graduate and forensic scientist Charlene Rittenbach is the first to greet us at the security entrance. We shuffle past a glass showcase of early-era breathalyzers and a collection of wildly artistic bongs no doubt, paraphernalia treasures seized from fantastic drug busts past into a large room that looks almost like any other business office. Its wall to wall cubicles and computers.

Rittenbach, who also serves as technical leader for the labs Forensic Chemistry Unit, tells us that this is where the scientists write their reports and review cases. Unlike some other crime labs, she explains, this one is all digital. Next, were introduced to Brian Herz and Marc Larson, two more forensic scientists and UND alums. We hear a few rules no bags, no liquids, no photos and then were off to the laboratories.

For the next several hours, were in a different world a fascinating world where hard science turns evidence into fact. Its a world where criminal prosecutors get the goods to prove guilt or innocence in illegal drug and DUI cases, arson, rape and any number of other assaults and untimely deaths.

I must admit for someone without a chemistry background, it felt as though everyone else was speaking a different language. Gas chromatography. Mass spectrometry. Liquid chromatography quadrupole time-of-flight mass spectrometry. Huh? Over and over again, as fast as my head was spinning, the students heads were nodding as they peppered the experts with questions.

Up close, we watched the so-called presumptive tests and confirmatory tests to identify all sorts of suspected illegal drugs. FYI: The top three in North Dakota are cannabis, methamphetamine and fentanyl.

The uptick in opioids and fentanyl compounds and new synthetic drugs in the state is just huge, Rittenbach said. Its like Whac-A-Mole just trying to keep up, and it wasnt that way with opioids just five years ago.

The most common way we see a lot of the opioids and fentanyl compounds now is in counterfeit oxycodone tablets, Herz said.

Then pointing to a teeny-tiny drug sample under a protective hood, he added: Its just crazy what people are putting into these things. You just think about how much is right there compared to a lethal dose. Its enough to kill everybody in this building at least four times over.

Thats all these drug dealers really care about is making money. They dont care who overdoses or dies because once that news gets out on the street, it actually makes the product even more valuable as messed up as that is. A lot of the users have been abusing opioids for years, so the heroin doesnt do it anymore. Theyre looking for something stronger.

Remember the elephants? Well, the latest trend for illegal drug dealers is to cut their opioids with veterinarian medicines such as the deadly large-mammal synthetic opioid called Carfentanil, and more recently with the tranquilizer Xylazine. FDA-approved as a veterinary medicine, Xylazine, known on the street as tranq or the zombie drug, extends the effects of fentanyl and mimics the high of heroin. Narcan cannot reverse a Xylazine overdose, yet users continue to crave the high even while suffering from the drugs other nasty side effect: persistent infections that rot their flesh to the bone.

Sometimes people dont even know this stuff is in there, Herz said. They can be at a party and think its just a regular oxycodone tablet, but its likely not. Your age group is their prime target. They want to get you hooked young. So, if you ever hear of anybody who has these (nonprescribed oxy) or wants to get some, I implore you to stop them or it could be the last time you see them alive. Whats out there now is very scary stuff.

Sobering advice for sure, but what the DNA experts had to say next was equally sobering.

Theres just something particularly discomforting about being asked to swipe a sterile cotton swab on the inside of your cheeks before sealing it inside an envelope with your name printed out front.

It wont go anywhere, I promise, Rittenbach assured us before we entered the DNA unit. We just need your sample for quality control in case some of your DNA would shed here and we would find a foreign profile in some of our evidence.

Ohhh OK, I guess that makes sense. Even so, most of us agreed it was an unsettling, eerie feeling to be sharing something so near and dear as our own personal DNA with a state crime laboratory. (I mean, really, in what other situation would this be a good thing?)

Next, we moved into the adjacent room, where we put on amber glasses and watched in the dark as forensic DNA experts demonstrated how different light sources can make otherwise invisible stains pop into view on a tattered black tank top. Thats blood. Thats bleach. Thats saliva. Thats semen. Eww.

In this case, the clothing and stains were used only for training exercises, but when investigating a real crime, the experts said they would darken the room lights to mark the stains and then cut small samples from the fabric always preserving enough DNA for the defense team to run their tests.

As though these demonstrations werent macabre enough, things were about to get more shocking as DNA analyst, forensic scientist and state CODIS Administrator Amy Gebhardt shared the case of (her) lifetime.

It was in the fall of 2006, Gebhardt began, when a Valley City, N.D., college student was found brutally killed in her off-campus apartment. Two friends had discovered her body with a belt around her neck and a broken knife in her throat.

As is common practice, the medical examiner had taken scrapings from beneath the victims fingernails.

Now Ive worked with fingernail scrapings my entire career, and most of the time, theres very little blood DNA from the perpetrator, Gebhardt explained. Its usually mostly from the victim, but thats what made this case so rare.

The preliminary tests were showing a male had contributed significantly to the blood sample. The lab then immediately entered the profile into the FBIs Combined DNA Index System, or CODIS, to see if it matched the DNA of any previous convicted offenders or arrest suspects. And they got a hit.

The DNA belonged to an unknown perpetrator who had raped a woman two years earlier in Fargo. That victim survived, but she could tell police little about her attacker other than his race.

The rest of the story is pretty crazy, Gebhardt went on to say as the wide-eyed students listened intently. She then explained how that following Monday, an overly curious man who lived in the same building as the victim had approached investigators who still were gathering evidence and taking pictures outside the apartment complex.

A man comes up to one of the BCI agents and says, So do you have any idea who this could be? Do you have any leads on who could have done such a horrible thing? Gebhardt relayed. And theyre like, Probably you. But they didnt say that.

Instead, they said, No, but were interested in collecting samples from people in the area who are willing to be elimination samples. And the guy actually said, Oh yeah, sure, no problem.

Thats what we count on: Stupidity! Its what I live for! These guys are so arrogant, and they think theyre so much better and so much smarter than everybody else, but its just beautiful.

The investigators collect his sample, along with elimination DNA from the victims dad, her boyfriend and a couple of other friends, then drive from Valley City to Bismarck to hand-deliver the samples to the lab by midafternoon.

Just three hours earlier, Gebhardts supervisor and then-CODIS director had left for a meeting in Arizona.

We knew they were driving these samples up, so I quit what I was doing and got them on the instrument right away. Its almost midnight. Im burned out. Im fried, but Im going to check the instrument one more time before I go home.

I look AND ITS HIM! IT MATCHES HIM! Oh my God! Im freaking out. Im alone in this small room. Im pacing. My hair on the back of my neck is standing up. Im shaking, and Im about to cry.

Moments later, Gebhardt wakes her boss with a phone call only to be directed to run the tests all over again. They must be certain none of the samples had gotten mixed up. Nope, by 4:30 the next morning, she gets the same results.

This was an especially horrific crime, and now they know who did it.

We said, This is your guy, Mo Maurice Gibbs. They arrested him by 6:30 that evening and had a TV press conference by 10:30 that night, she said.

Come to find out, she added, the suspect coached youth basketball and also worked at the jail, where he later was accused and convicted of assaulting several female inmates. He was shutting off the cameras to their cells.

Her voice cutting out and now in tears, Gebhardt tells the students, Thats why I say this is a case of a lifetime. I guess my reason for sharing this case is to tell you that all these little steps so clinical and sterile, piece by piece, the data, the profiles, the peak heights and all the statistics sometimes it all comes together so beautifully.

Then, almost in a single breath, she blurts out: This job, Im going to tell you, is dark. You should not be looking for semen in a diaper or on an Ariel, Little Mermaid blanket. There are horrible things that happen in this state. Do not let our general good nature and low crime statistics fool you. Bad things that never make the news are happening in this state all the time. This is real life. Ive been here for 25 years, and there are days Im burnt out.

I was probably like you in my 20s, morbidly curious. I was Yes, give me all of it. But it wears on you, and thats not something I expected. Thats why Im telling you to be prepared. Keep your mental health a priority. Find ways to rejuvenate, recharge and keep perspective because in this job, you are immersed 100 percent of the time in the worst of humanity. It is rewarding and it is wonderful, but IT IS DARK.

Finally, pausing to catch her breath, she asks, Do you have any questions?

Somber and silent, the students have nothing to say.

I honestly didnt know what to expect from the field trip, but it certainly was eye-opening.

On the long ride home, Iancu tells me shes always looking for ways to add value to the student experience.

I love what I do and I can teach my courses, but we really have to think further than the academic training. What are my students going to do after they graduate? I feel responsible for their future, she said. Im always thinking How can I improve? What can I add? What types of scholarships are out there? What sort of collaborations can I find?

I want my students to be able to network and make these connections. The most important goal for me with the crime lab and other hands-on experiences is to give these students their best chance to get hired in this profession.

And what insight did the four students Monatukwa, Wellens, Eviota and Ras gain from hearing about real-life crime cases and seeing forensic science in action?

Monatukwa said it was extremely valuable to learn what the labs are looking for in potential hires as far as specific education and training not to mention, she was able to compile some great questions for her next-day interview in Colorado. And as an undergrad teaching assistant, she also was keenly interested in how the experts communicated the science to people with different levels of background and understanding.

Im trying to learn how to be a teacher at the same time Im trying to learn how to be a forensic scientist, so watching how they might explain something to a layperson, or jury, was fascinating, Monatukwa said. I think you actually added value by being here and representing that layperson. Plus, all kinds of cross-networking happened on this trip, and I really enjoyed that.

For Wellens: People learn in a lot of different ways, and I think the hands-on experience can help a lot of people better grasp what theyre studying. Its kind of a different experience when youre actually doing the work vs. just reading about the work. It makes me more excited to get to work in the lab setting.

Eviota agreed: I thought it was really cool to see how this CSI stuff actually works. In my mind, I was still thinking of everything in kind of sci-fi style with these big machines doing a lot of different things. This brought everything down to earth for me.

And Ras summed it up this way: Today was so much fun and so inspiring. Everyone was so passionate about what they do, and it made me more passionate. Were so lucky to have someone like Lavinia whos pulling strings everywhere to get students experiences theyd never get anywhere else. I absolutely loved it. Today just solidified how awesome forensic science is.

READ PAST FORENSIC SCIENCE STORIES: A little fly told herand Was it the knife, the candlestick or the rope?

Charlene Rittenbach points out some visual cues often spotted by investigators when trying to first identify suspected illegal drugs. Photo by Janelle Vonasek/UND Today.

Continue reading here:
From clues to facts to convictions - UND Today - University of North Dakota

Inside the largely unregulated market for bodies donated to science: "It’s harder to sell hot dogs on a cart" – CBS News

Watch theCBS Reportsdocumentary "Body Brokers" in the video player above. It streams live on the CBS News app at 8p, 11p and 2a ET on your mobile or streaming device.

Every year, an estimated 20,000 people donate their bodies to science for the purpose of medical research and education. But unlike organ donation, these body parts can be bought and sold for profit a market with very few federal regulations.

Special Agent Paul Micah Johnson, an investigator for the FBI's Detroit Division, has spent the last decade looking into what he called a "vast gray and black market of dead human bodies."

click to expand

The Uniform Anatomical Gift Act, a statute adopted by 47 states, is one of the few regulations governing the non-transplant tissue industry. It outlines the standards for body donation, and most significantly, it requires that donors, or their loved ones, must provide informed consent before donation takes place.

These consent agreements between donors and tissue banks typically stipulate that the body will be used for research and education but what that means in practice can vary.

"Medical research and education, particularly education, is a vague term and it is not clearly defined even in the Uniform Anatomical Gift Act," says Johnson. "The misleading of families across the industry is quite common."

While most donors consent to donation for research and education, legally, the agreements can include addendums that allow for donors to be used for "non-medical projects" that can include "crime scene investigation" and "vehicle safety."

Steve Hansen had always wanted to be an organ donor, but when he died in 2012 from cirrhosis of the liver, doctors said his organs were not healthy enough to be transplanted. Steve's wife Jill Hansen says hospice workers at the time suggested whole body donation as an alternative.

"What I envisioned was him being in some medical facility," Hansen says. "I just thought, what a great candidate for them to learn about the results of alcoholism and what it does to a body."

Steve's body was eventually sent to the Biological Resource Center in Phoenix, Arizona, which then sold his remains to the Department of Defense.

"They told me specifically that my husband had been used as a crash test dummy in a simulated Humvee explosion," Hansen says.

Hansen's husband's body was sold by Biological Resource Center founder Stephen Gore to the DOD without her consent. Court records showed that those donors were used in a variety of military and ballistics testing, with some resulting in "the complete mutilation and desecration of the donor's body."

"I was devastated," Hansen says. "I would've never done it if I had known. I just kept telling him I was sorry."

When the FBI raided Gore's warehouse in 2014, they found conditions of the donors' remains so abhorrent that agents "required trauma therapy due to the disturbing, graphic scene they encountered."

Gore pleaded guilty to Illegal Control of an Enterprise for violating donor consent agreements and was sentenced to four years probation and one year in jail. Because there are few laws defining ethical treatment of remains, no charges were filed relating to the warehouse conditions.

Disturbing evidence also emerged during Special Agent Johnson's investigation into body broker Arthur Rathburn, whose Detroit warehouse was later described by investigators as a "house of horrors."

"He cut up bodies with a chainsaw. He cut up bodies with a bandsaw," Johnson says. "He had a bucket filled with brown liquid that had fetuses and human brains floating in it."

As in the case with Gore, prosecutors had no way of charging Rathburn for how donors' bodies were treated. Rathburn was ultimately sentenced in 2018 to nine years in prison for falsifying his donor's medical information in order to fraudulently sell bodies infected with hepatitis and HIV.

"He had trash cans filled with human heads," Johnson says. "Just the disrespectful way of treating the remains. And yet we prosecuted him for fraud."

In 2009, Donor Referral Services owner Philip Guyett Jr., pleaded guilty to wire fraud for falsifying medical information in order to offload tissue designated for transplant with infectious or communicable diseases. He was sentenced to eight years in prison.

"As I told the judge, I had no business being in this business," Guyett says in an interview with CBS News. "A person with no medical experience, no funeral director's license, was able to open up a whole body donation program, take possession of a human body, dismember it, send it throughout the nation without any type of licensing oversight. It's harder to sell hot dogs on a cart than it is to get into this business."

Before opening his own tissue bank, Guyett got his start running a university's Willed Body Program, overseeing the donations of bodies for research education.

In the wake of industry controversies, some tissue bankers have been working to promote trust with potential donors.

Garland Shreves is the founder of Research For Life, a non-transplant tissue bank based in Phoenix that distributes body parts for use towards the advancement of science in lifesaving fields like first-responder training, medical device development and surgical practice.

"I don't think we can ever lose sight of the fact that these are human beings and they have so graciously donated their body to the advancement of medicine," Shreves says. "Whole body donation and my organization, through the wonderful gift of that donor, makes it possible for us to have the innovative medicine that we do today in this country."

Shreves says the best way to clean up the industry is transparency.

"Bad actors are bad for business. If the consumer is being harmed by a business that's not operating at a very high level of professionalism, everyone in the industry suffers," he says. "It is that transparency, the sunlight, the disinfectant, if you will. It's what we need to do as an industry and we haven't done enough of it."

Johnson says the whole body donation industry is necessary for the advancement of science, and that it is vital the industry works to restore the public's trust.

"It would be nice if there was one playbook for everyone. And so that would ideally be federal and it would cover everyone that deals with human body parts for-profit, nonprofit, all of them under one set of rules."

In September 2022, a body broker bill was introduced in the U.S. Senate which would impose federal regulations on the process of body donation, but no vote has been scheduled.

Original post:
Inside the largely unregulated market for bodies donated to science: "It's harder to sell hot dogs on a cart" - CBS News

Defense motions to hire expert witnesses in Fairfield murder case – Southeast Iowa Union

FAIRFIELD The defense team of Fairfield teen Willard Noble Chaiden Miller has asked the government to pay for a slate of expert witnesses that it plans to call during Millers murder trial in April.

However, prosecutors in the case are resisting this attempt, arguing that hiring these witnesses at such a late stage in the process would require delaying the trial even further. Miller, 17, is being charged with the murder of Nohema Graber, 66, who was a Spanish teacher at Fairfield High School at the time of her death in November 2021.

District Court Judge Shawn Showers has agreed to a hearing on the recent motion by the defense, and has set a date of March 29 for the Jefferson County Courthouse. Miller is scheduled to stand trial starting April 21 in Council Bluffs.

Millers defense team, led by attorney Christine Branstad, has requested the state pay for expert witnesses such as: a crime scene analyst for up to $10,000; a digital forensic expert for $9,000; a social psychologist for $7,000; a clinical psychologist for $5,000; and an investigator for $3,500.

Branstad stated in her motion that her client was found indigent by the court, meaning he could not pay for his own expenses. She stated that a digital forensic expert was necessary since the state intends to offer into evidence cellphone information obtained from Miller and his co-defendant, Jeremy Everett Goodale, who is also charged with murder.

Branstad argued that a social psychologist was necessary to address questions about a persons memory and its limitations, and how it could be affected by alcohol or other substances.

In filing his resistance to Branstads motion, Jefferson County Attorney Chauncey Moulding argued that the court should deny the motion for its untimeliness, because the defense has had access to crime scene investigation and digital evidence from cellphones for more than a year. He said that was true for other pieces of evidence, too.

Both the social psychologist and clinical psychologist testimony, assuming it is allowed, would impact evidence that has been known to the defendant since the inception of the case, Moulding wrote. Based on their motion, it appears no work has been completed by the defenses requested witnesses.

Moulding also expressed his doubt that so many expert witnesses could be available on such short notice, since the trial is just a month away.

There is very little time to conduct the necessary review and depositions of the witnesses, Moulding wrote. The State intends to make every effort to complete discovery of the witnesses listed by the defense in order to be ready for trial on April 21, 2023.

Call Andy Hallman at 641-575-0135 or email him at andy.hallman@southeastiowaunion.com

View post:
Defense motions to hire expert witnesses in Fairfield murder case - Southeast Iowa Union

Learning to grow machine-learning models | MIT News | Massachusetts Institute of Technology – MIT News

Its no secret that OpenAIs ChatGPT has some incredible capabilities for instance, the chatbot can write poetry that resembles Shakespearean sonnets or debug code for a computer program. These abilities are made possible by the massive machine-learning model that ChatGPT is built upon. Researchers have found that when these types of models become large enough, extraordinary capabilities emerge.

But bigger models also require more time and money to train. The training process involves showing hundreds of billions of examples to a model. Gathering so much data is an involved process in itself. Then come the monetary and environmental costs of running many powerful computers for days or weeks to train a model that may have billions of parameters.

Its been estimated that training models at the scale of what ChatGPT is hypothesized to run on could take millions of dollars, just for a single training run. Can we improve the efficiency of these training methods, so we can still get good models in less time and for less money? We propose to do this by leveraging smaller language models that have previously been trained, says Yoon Kim, an assistant professor in MITs Department of Electrical Engineering and Computer Science and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL).

Rather than discarding a previous version of a model, Kim and his collaborators use it as the building blocks for a new model. Using machine learning, their method learns to grow a larger model from a smaller model in a way that encodes knowledge the smaller model has already gained. This enables faster training of the larger model.

Their technique saves about 50 percent of the computational cost required to train a large model, compared to methods that train a new model from scratch. Plus, the models trained using the MIT method performed as well as, or better than, models trained with other techniques that also use smaller models to enable faster training of larger models.

Reducing the time it takes to train huge models could help researchers make advancements faster with less expense, while also reducing the carbon emissions generated during the training process. It could also enable smaller research groups to work with these massive models, potentially opening the door to many new advances.

As we look to democratize these types of technologies, making training faster and less expensive will become more important, says Kim, senior author of a paper on this technique.

Kim and his graduate student Lucas Torroba Hennigen wrote the paper with lead author Peihao Wang, a graduate student at the University of Texas at Austin, as well as others at the MIT-IBM Watson AI Lab and Columbia University. The research will be presented at the International Conference on Learning Representations.

The bigger the better

Large language models like GPT-3, which is at the core of ChatGPT, are built using a neural network architecture called a transformer. A neural network, loosely based on the human brain, is composed of layers of interconnected nodes, or neurons. Each neuron contains parameters, which are variables learned during the training process that the neuron uses to process data.

Transformer architectures are unique because, as these types of neural network models get bigger, they achieve much better results.

This has led to an arms race of companies trying to train larger and larger transformers on larger and larger datasets. More so than other architectures, it seems that transformer networks get much better with scaling. Were just not exactly sure why this is the case, Kim says.

These models often have hundreds of millions or billions of learnable parameters. Training all these parameters from scratch is expensive, so researchers seek to accelerate the process.

One effective technique is known as model growth. Using the model growth method, researchers can increase the size of a transformer by copying neurons, or even entire layers of a previous version of the network, then stacking them on top. They can make a network wider by adding new neurons to a layer or make it deeper by adding additional layers of neurons.

In contrast to previous approaches for model growth, parameters associated with the new neurons in the expanded transformer are not just copies of the smaller networks parameters, Kim explains. Rather, they are learned combinations of the parameters of the smaller model.

Learning to grow

Kim and his collaborators use machine learning to learn a linear mapping of the parameters of the smaller model. This linear map is a mathematical operation that transforms a set of input values, in this case the smaller models parameters, to a set of output values, in this case the parameters of the larger model.

Their method, which they call a learned Linear Growth Operator (LiGO), learns to expand the width and depth of larger network from the parameters of a smaller network in a data-driven way.

But the smaller model may actually be quite large perhaps it has a hundred million parameters and researchers might want to make a model with a billion parameters. So the LiGO technique breaks the linear map into smaller pieces that a machine-learning algorithm can handle.

LiGO also expands width and depth simultaneously, which makes it more efficient than other methods. A user can tune how wide and deep they want the larger model to be when they input the smaller model and its parameters, Kim explains.

When they compared their technique to the process of training a new model from scratch, as well as to model-growth methods, it was faster than all the baselines. Their method saves about 50 percent of the computational costs required to train both vision and language models, while often improving performance.

The researchers also found they could use LiGO to accelerate transformer training even when they didnt have access to a smaller, pretrained model.

I was surprised by how much better all the methods, including ours, did compared to the random initialization, train-from-scratch baselines. Kim says.

In the future, Kim and his collaborators are looking forward to applying LiGO to even larger models.

The work was funded, in part, by the MIT-IBM Watson AI Lab, Amazon, the IBM Research AI Hardware Center, Center for Computational Innovation at Rensselaer Polytechnic Institute, and the U.S. Army Research Office.

See the original post here:
Learning to grow machine-learning models | MIT News | Massachusetts Institute of Technology - MIT News

Dense reinforcement learning for safety validation of autonomous vehicles – Nature.com

Kalra, N. & Paddock, S. M. Driving to safety: how many miles of driving would it take to demonstrate autonomous vehicle reliability? Transp. Res. A 94, 182193 (2016).

Google Scholar

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436444 (2015).

Article ADS CAS PubMed Google Scholar

10 million self-driving cars will be on the road by 2020. Insider https://www.businessinsider.com/report-10-million-self-driving-cars-will-be-on-the-road-by-2020-2015-5-6 (2016).

Nissan promises self-driving cars by 2020. Wired https://www.wired.com/2013/08/nissan-autonomous-drive/ (2014).

Teslas self-driving vehicles are not far off. Insider https://www.businessinsider.com/elon-musk-on-teslas-autonomous-cars-2015-9 (2015).

Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles (Society of Automotive Engineers, 2021); https://www.sae.org/standards/content/j3016_202104/.

2021 Disengagement Reports (California Department of Motor Vehicles, 2022); https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/disengagement-reports/.

Paz, D., Lai, P. J., Chan, N., Jiang, Y. & Christensen, H. I. Autonomous vehicle benchmarking using unbiased metrics. In IEEE International Conference on Intelligent Robots and Systems 62236228 (IEEE, 2020).

Favar, F., Eurich, S. & Nader, N. Autonomous vehicles disengagements: trends, triggers, and regulatory limitations. Accid. Anal. Prev. 110, 136148 (2018).

Article PubMed Google Scholar

Riedmaier, S., Ponn, T., Ludwig, D., Schick, B. & Diermeyer, F. Survey on scenario-based safety assessment of automated vehicles. IEEE Access 8, 8745687477 (2020).

Article Google Scholar

Nalic, D. et al. Scenario based testing of automated driving systems: a literature survey. In Proc. of the FISITA Web Congress 110 (Fisita, 2020).

Feng, S., Feng, Y., Yu, C., Zhang, Y. & Liu, H. X. Testing scenario library generation for connected and automated vehicles, part I: methodology. IEEE Trans. Intell. Transp. Syst. 22, 15731582 (2020).

Article Google Scholar

Feng, S. et al. Testing scenario library generation for connected and automated vehicles, part II: case studies. IEEE Trans. Intell. Transp. Syst. 22, 56355647 (2020).

Article Google Scholar

Feng, S., Yan, X., Sun, H., Feng, Y. & Liu, H. X. Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment. Nat. Commun. 12, 748 (2021).

Article ADS CAS PubMed PubMed Central Google Scholar

Sinha, A., OKelly, M., Tedrake, R. & Duchi, J. C. Neural bridge sampling for evaluating safety-critical autonomous systems. Adv. Neural Inf. Process. Syst. 33, 64026416 (2020).

Google Scholar

Li, L. et al. Parallel testing of vehicle intelligence via virtual-real interaction. Sci. Robot. 4, eaaw4106 (2019).

Article PubMed Google Scholar

Zhao, D. et al. Accelerated evaluation of automated vehicles safety in lane-change scenarios based on importance sampling techniques. IEEE Trans. Intell. Transp. Syst. 18, 595607 (2016).

Article PubMed PubMed Central Google Scholar

Donoho, D. L. High-dimensional data analysis: the curses and blessings of dimensionality. AMS Math Challenges Lecture 1, 32 (2000).

Google Scholar

Hinton, G. E. & Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science 313, 504507 (2006).

Article ADS MathSciNet CAS PubMed MATH Google Scholar

Silver, D. et al. Mastering the game of go without human knowledge. Nature 550, 354359 (2017).

Article ADS CAS PubMed Google Scholar

Mirhoseini, A. et al. A graph placement methodology for fast chip design. Nature 594, 207212 (2021).

Article ADS CAS PubMed Google Scholar

Cummings, M. L. Rethinking the maturity of artificial intelligence in safety-critical settings. AI Mag. 42, 615 (2021).

Google Scholar

Kato, S. et al. Autoware on board: enabling autonomous vehicles with embedded systems. In 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems 287296 (IEEE, 2018).

Feng, S. et al. Safety assessment of highly automated driving systems in test tracks: a new framework. Accid. Anal. Prev. 144, 105664 (2020).

Article PubMed Google Scholar

Lopez, P. et al. Microscopic traffic simulation using SUMO. In International Conference on Intelligent Transportation Systems 25752582 (IEEE, 2018).

Arun, A., Haque, M. M., Bhaskar, A., Washington, S. & Sayed, T. A systematic mapping review of surrogate safety assessment using traffic conflict techniques. Accid. Anal. Prev. 153, 106016 (2021).

Article PubMed Google Scholar

Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 2018).

Koren, M., Alsaif, S., Lee, R. & Kochenderfer, M. J. Adaptive stress testing for autonomous vehicles. In IEEE Intelligent Vehicles Symposium (IV) 17 (IEEE, 2018).

Sun, H., Feng, S., Yan, X. & Liu, H. X. Corner case generation and analysis for safety assessment of autonomous vehicles. Transport. Res. Rec. 2675, 587600 (2021).

Article Google Scholar

Schulman, J., Wolski, F., Dhariwal, P., Radford, A. & Klimov, O. Proximal policy optimization algorithms. Preprint at https://arxiv.org/abs/1707.06347 (2017).

Owen, A. B. Monte Carlo theory, methods and examples. Art Owen https://artowen.su.domains/mc/ (2013).

Krajewski, R., Moers, T., Bock, J., Vater, L. & Eckstein, L. September. The round dataset: a drone dataset of road user trajectories at roundabouts in Germany. In 2020 IEEE 23rd International Conference on Intelligent Transportation Systems 16 (IEEE, 2020).

Nowakowski, C., Shladover, S. E., Chan, C. Y. & Tan, H. S. Development of California regulations to govern testing and operation of automated driving systems. Transport. Res. Rec. 2489, 137144 (2015).

Article Google Scholar

Sauerbier, J., Bock, J., Weber, H. & Eckstein, L. Definition of scenarios for safety validation of automated driving functions. ATZ Worldwide 121, 4245 (2019).

Article Google Scholar

Pek, C., Manzinger, S., Koschi, M. & Althoff, M. Using online verification to prevent autonomous vehicles from causing accidents. Nat. Mach. Intell. 2, 518528 (2020).

Article Google Scholar

Seshia, S. A., Sadigh, D. & Sastry, S. S. Toward verified artificial intelligence. Commun. ACM 65, 4655 (2022).

Article Google Scholar

Wing, J. M. A specifiers introduction to formal methods. IEEE Comput. 23, 824 (1990).

Article Google Scholar

Li, A., Sun, L., Zhan, W., Tomizuka, M. & Chen, M. Prediction-based reachability for collision avoidance in autonomous driving. In 2021 IEEE International Conference on Robotics and Automation 79087914 (IEEE, 2021).

Automated Vehicle Safety Consortium AVSC Best Practice for Metrics and Methods for Assessing Safety Performance of Automated Driving Systems (ADS) (SAE Industry Technologies Consortia, 2021).

Au, S. K. & Beck, J. L. Important sampling in high dimensions. Struct. Saf. 25, 139163 (2003).

Article Google Scholar

Silver, D., Singh, S., Precup, D. & Sutton, R. S. Reward is enough. Artif. Intell. 299, 113 (2021).

Article MathSciNet MATH Google Scholar

Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529533 (2015).

Article ADS CAS PubMed Google Scholar

Weng, B., Rao, S. J., Deosthale, E., Schnelle, S. & Barickman, F. Model predictive instantaneous safety metric for evaluation of automated driving systems. In IEEE Intelligent Vehicles Symposium (IV) 18991906 (IEEE, 2020).

Junietz, P., Bonakdar, F., Klamann, B. & Winner, H. Criticality metric for the safety validation of automated driving using model predictive trajectory optimization. In International Conference on Intelligent Transportation Systems 6065 (IEEE, 2018).

Huang, G., Liu, Z., Van Der Maaten, L. & Weinberger, K. Q. Densely connected convolutional networks. In IEEE Conference on Computer Vision and Pattern Recognition 47004708 (IEEE, 2017).

Bengio, Y., Louradour, J., Collobert, R. & Weston, J. Curriculum learning. In International Conference on Machine Learning 4148 (ICML, 2009).

Yan, X., Feng, S., Sun, H., & Liu, H. X. Distributionally consistent simulation of naturalistic driving environment for autonomous vehicle testing. Preprint at https://arxiv.org/abs/2101.02828 (2021).

Bezzina, D. & Sayer, J. Safety Pilot Model Deployment: Test Conductor Team Report DOT HS 812 171 (National Highway Traffic Safety Administration, 2014).

Sayer, J. et al. Integrated Vehicle-based Safety Systems Field Operational Test: Final Program Report FHWA-JPO-11-150; UMTRI-2010-36 (Joint Program Office for Intelligent Transportation Systems, 2011).

Treiber, M., Hennecke, A. & Helbing, D. Congested traffic states in empirical observations and microscopic simulations. Phys. Rev. E 62, 1805 (2000).

Article ADS CAS MATH Google Scholar

Kesting, A., Treiber, M. & Helbing, D. General lane-changing model MOBIL for car-following models. Transp. Res. Rec. 1999, 8694 (2007).

Article Google Scholar

Liang, E. et al. RLlib: abstractions for distributed reinforcement learning. In International Conference on Machine Learning 30533062 (ICML, 2018).

Chang A. X. et al. ShapeNet: an information-rich 3D model repository. Preprint at https://arxiv.org/abs/1512.03012 (2015).

Darweesh, H. et al. Open source integrated planner for autonomous navigation in highly dynamic environments. J. Robot. Mechatron. 29, 668684 (2017).

Article Google Scholar

View original post here:
Dense reinforcement learning for safety validation of autonomous vehicles - Nature.com