Brian Reggiannini figures out who’s talking

If computers could become ‘smart’ enough to recognize who is talking, that could allow them to produce real-time transcripts of meetings, courtroom proceedings, debates, and other important events. In the dissertation that will allow him to receive his Ph.D. at Commencement this year, Brian Reggiannini found a way to advance the state of the art for voice- and speaker-recognition.

Real-time tracking of who’s talking
With the right algorithms and signal processing software, an
array of button-size microphones placed around the perimeter
of a room can identify, follow, and record each of several
people as they move about, interrupt each other, and converse.
Credit: Frank Mullin/Brown University

Everyone does signal processing every day, even if we don’t call it that. With friends at a sports bar, we peer up at the TV to see the score, we turn our head toward the crashing sound when a waitress drops a glass, and perhaps most remarkably, we can track the fast-paced banter of all the people in our booth, even if we’ve never met some of the friends-of-friends who have insinuated themselves into the scene.

Very few of us, however, could ever get a computer to do anything like that. That’s why doing it well has earned Brian Reggiannini a Ph.D. at Brown and a career in the industry.

In his dissertation, Reggiannini managed to raise the bar for how well a computer connected to a roomful of microphones can keep track of who among a small group of speakers is talking. Further refined and combined with speech recognition, such a system could lead to instantaneous transcriptions of meetings, courtroom proceedings, or debates among, say, several rude political candidates who are prone to interrupt. It could help the deaf follow conversations in real-time.

If only it weren’t so hard to do.

Brian Reggiannini
“We’re trying to teach a computer how to do something that we as humans do so naturally that we don’t even understand how we do it.”

But Reggiannini, who came to Brown as an undergraduate in 2003 and began building microphone arrays in the lab of Harvey Silverman, professor of engineering, in his junior year, was determined to advance the state of the art.

The specific challenge he set for himself was real-time tracking of who’s talking among at least a few people who are free to rove around a room. Hardware was not the issue. The test room on campus has 448 microphones all around the walls and he only used 96. That was enough to gather the kind of information that allows systems – think of your two ears – to locate the source of a sound.

The real rub was in devising the algorithms and, more abstractly, in realizing where his reasoning about the problem had to abandon the conventional wisdom.

Previous engineers who had tried something like this were on the right track. After all, there is only so much data available in situations like this. Some tried analyzing accents, pronunciation, word use, and cadence, but those are complex to track and require a lot of data. The simpler features are pitch, volume, and spectral statistics (a breakdown of a voice’s component waves and frequencies) of each speaker’s voice. Systems can also ascertain where a voice came from within the room.

Snippets, not speakers

But many attempts to build speaker identification systems (like the voice recognition in your personal computer) have relied on the idea that a computer could be extensively trained in “clean,” quiet conditions to learn a speaker’s voice in advance.

One of Reggiannini’s key insights was that just like a politician couldn’t possibly be primed to recognize every voter at a rally, it’s unrealistic to train a speaker-recognition system with the voice of everyone who could conceivably walk into a room.

Instead, Reggiannini sought to build a system that could learn to distinguish the voices of anyone within a session. It analyzes each new segment of speech and also notes the distinct physical position of individuals within the room. The system compares each new segment, or snippet, of what it hears to previous snippets. It then determines a statistical likelihood that the new snippet would have come from a speaker it has already identified as unique.

“Instead of modeling talkers, I’m going to instead model pairs of speech segments,” Reggiannini recalled.

A key characteristic of Reggiannini’s system is that it can work with very short snippets of speech. It doesn’t need full sentences to work at least somewhat well. That’s important because it’s realistic. People don’t speak in florid monologues. They speak in fractured conversations. No way! Yes, really.

People also are known to move around. For that reason position as inferred by the array of microphones can be only an intermittent asset. At any single moment in time, especially at the beginning of a session, position helpfully distinguishes each talker from every other (no two people can be in the same place at the same time), but when people stop talking and start walking, the system necessarily loses track of them until they speak again.

Reggiannini tested his system every step of the way. His experiments included just pitch analysis, just spectral analysis, a combination of the two, position alone, and a combination of the full speech analysis and position tracking. He subjected the system to a multitude of voices, sometimes male-only, sometimes female-only, and sometimes mixed. In every case, at least until the speech snippets became quite long, his system was better able to discriminate among talkers than two other standard approaches.

That said, the system sometimes is uncertain and in cases like that it defers assigning speech to a talker until it is more certain. Once it is, it goes back and labels the snippets accordingly.

It’s no surprise that the system would err, or hedge, here and there. Reggiannini’s test room was noisy. While some systems are fed very clean audio, the only major concessions that Reggiannini allowed himself were that speakers wouldn’t run or jump across the room and that only one would speak from the script at a time. The ability to filter individual voices out from within overlapping speech is perhaps the biggest remaining barrier between the system remaining a research project and becoming a commercial success.

A career in the field

While the ultimate fate of Reggiannini’s innovations is not yet clear, what is certain is that he has been able to embark on a career in the field he loves. Since leaving Brown last summer he’s been working as a digital signal processing engineer at Analog Devices in Norwood, Mass., which happens to be his hometown.

Reggiannini has yet to work on an audio project, but that’s fine with him. His interest is the signal processing, not sound per se. Instead he’s applied his expertise to challenges of heart monitoring and wireless communications.

“I’ve been jumping around applications but all the fundamental signal processing theory applies no matter what the signal is,” he said. “My background lets me work on a wide range of problems.”

After seven years and three degrees at Brown, Reggiannini was prepared to pursue his passion.

- by David Orenstein

Read More

Brown Team Wins Rhode Island Business Plan Competition For Fourth Straight Year

For the fourth consecutive year, a Brown student team has won the Rhode Island Business Plan Competition. This year, Overhead.fm, led by Stephen Hebson ’12, an economics and history double concentrator, and Parker Wells ’12, a mechanical engineering concentrator, won the student track of the competition. Their plan is to produce a web app to provide a customizable stream of music for in-store use, licensing music directly from the artists and labels. They received $15,000 in cash and services valued at $24,000 for winning the competition. Winners were announced at the annual RIBX business expo at the Rhode Island Convention Center. In order to be eligible to win, applicants had to agree to establish or maintain operations in Rhode Island.

Previous Brown winners of the competition have included PriWater, now Premama, (2011), Speramus (2010), and Runa (2009). Premama (http://drinkpremama.com/) produces a prenatal beverage supplement to help reduce birth defects. Speramus is (www.speramus.com) an online fundraising platform that matches donors with individual support opportunities. Runa (www.runa.org) produces energy drinks made from the leaves of an Amazonian tree, and has been featured in the New York Times and raised over $1 million from investors and has increased distribution into Whole Foods.

Two other Brown teams, both with connections to the School of Engineering, were also named finalists at the 2012 competition. Finalists receive $5,000 in cash and services valued at between $9,000 and $11,500.

One of the finalists, JCD Wind, was from Steve Petteruti’s Entrepreneurship I class and Entrepreneurship II classes, Engineering 1930G and Engineering 1930H. JCD Wind, included James McGinn ’12, a biomedical engineering concentrator, and Carli Wiesenfeld ’12, a commerce, organizations, and entrepreneurship (COE) concentrator. The company aims to make seamless, high strength lightweight carbon fiber turbine blades. McGinn is also a member of the men’s rugby team, and Wiesenfeld is a member of the women’s gymnastics team.

The other finalist was a group of four graduate students in the Program in Innovation Management and Entrepreneurship (PRIME) master’s degree program. Solar4Cents included PRIME students Sean Pennino, Bhavuk Nagpal, Xiaotong 'Peter' Shan, and Meng 'Milo' Zhang. Solar4Cents is a manufacturing company that aims to produce low-cost, thin-film copper, zinc, tin, and sulfur solar cells for solar panel manufacturers.

Established in 2000, the Rhode Island Business Plan Competition has awarded more than $1 million in prizes since its inception.

For the official RI Business plan release on the competition, please go to:
http://www.ri-bizplan.com/tabid/259/Default.aspx

Read More

Single nanomaterial yields many laser colors

Engineers at Brown University and QD Vision Inc. have created nanoscale single crystals that can produce the red, green, or blue laser light needed in digital displays. The size determines color, but all the pyramid-shaped quantum dots are made the same way of the same elements. In experiments, light amplification required much less power than previous attempts at the technology. The team’s prototypes are the first lasers of their kind.

PROVIDENCE, R.I. [Brown University] — Red, green, and blue lasers have become small and cheap enough to find their way into products ranging from BluRay DVD players to fancy pens, but each color is made with different semiconductor materials and by elaborate crystal growth processes. A new prototype technology demonstrates all three of those colors coming from one material. That could open the door to making products, such as high-performance digital displays, that employ a variety of laser colors all at once.

Vertical-cavity surface-emitting laser
Colloidal quantum dots — nanocrystals — can produce lasers of
many colors. Cuong Dang manipulates a green beam that pumps
the nanocrystals with energy, in this case producing red laser
light. Credit: Mike Cohea/Brown University

“Today in order to create a laser display with arbitrary colors, from white to shades of pink or teal, you’d need these three separate material systems to come together in the form of three distinct lasers that in no way shape or form would have anything in common,” said Arto Nurmikko, professor of engineering at Brown University and senior author of a paper describing the innovation in the journal Nature Nanotechnology. “Now enter a class of materials called semiconductor quantum dots.”

The materials in prototype lasers described in the paper are nanometer-sized semiconductor particles called colloidal quantum dots or nanocrystals with an inner core of cadmium and selenium alloy and a coating of zinc, cadmium, and sulfur alloy and a proprietary organic molecular glue. Chemists at QD Vision of Lexington, Mass., synthesize the nanocrystals using a wet chemistry process that allows them to precisely vary the nanocrystal size by varying the production time. Size is all that needs to change to produce different laser light colors: 4.2 nanometer cores produce red light, 3.2 nanometer ones emit green light and 2.5 nanometer ones shine blue. Different sizes would produce other colors along the spectrum.

The cladding and the nanocrystal structure are critical advances beyond previous attempts to make lasers with colloidal quantum dots, said lead author Cuong Dang, a senior research associate and nanophotonics laboratory manager in Nurmikko’s group at Brown. Because of their improved quantum mechanical and electrical performance, he said, the coated pyramids require 10 times less pulsed energy or 1,000 times less power to produce laser light than previous attempts at the technology.

Quantum nail polish
When chemists at QDVision brew a batch of colloidal quantum dots for Brown-designed specifications, Dang and Nurmikko get a vial of a viscous liquid that Nurmikko said somewhat resembles nail polish. To make a laser, Dang coats a square of glass — or a variety of other shapes — with the liquid. When the liquid evaporates, what’s left on the glass are several densely packed solid, highly ordered layers of the nanocrystals. By sandwiching that glass between two specially prepared mirrors, Dang creates one of the most challenging laser structures, called a vertical-cavity surface-emitting laser. The Brown-led team was the first to make a working VCSEL with colloidal quantum dots.

The nanocrystals’ outer coating alloy of zinc, cadmium, sulfur and that molecular glue is important because it reduces an excited electronic state requirement for lasing and protects the nanocrystals from a kind of crosstalk that makes it hard to produce laser light, Nurmikko said. Every batch of colloidal quantum dots has a few defective ones, but normally just a few are enough to interfere with light amplification.

Faced with a high excited electronic state requirement and destructive crosstalk in a densely packed layer, previous groups have needed to pump their dots with a lot of power to push them past a higher threshold for producing light amplification, a core element of any laser. Pumping them intensely, however, gives rise to another problem: an excess of excited electronic states called excitons. When there are too many of these excitons among the quantum dots, energy that could be producing light is instead more likely to be lost as heat, mostly through a phenomenon known as the Auger process.

The nanocrystals’ structure and outer cladding reduces destructive crosstalk and lowers the energy needed to get the quantum dots to shine. That reduces the energy required to pump the quantum dot laser and significantly reduces the likelihood of exceeding the level of excitons at which the Auger process drains energy away. In addition, a benefit of the new approach’s structure is that the dots can act more quickly, releasing light before Auger process can get started, even in the rare cases when it still does start.

“We have managed to show that it’s possible to create not only light, but laser light,” Nurmikko said. “In principle, we now have some benefits: using the same chemistry for all colors, producing lasers in a very inexpensive way, relatively speaking, and the ability to apply them to all kinds of surfaces regardless of shape. That makes possible all kinds of device configurations for the future.”

In addition to Nurmikko and Dang, another author at Brown is Joonhee Lee. QD Vision authors include Craig Breen, Jonathan Steckel, and Seth Coe-Sullivan, a company co-founder who studied engineering at Brown as an undergraduate.

The US. Department of Energy, the Air Force Office for Scientific Research, and the National Science Foundation supported the research. Dang is a Vietnam Education Foundation (VEF) Scholar.

Read More

Three Brown Teams Named Finalists in Rhode Island Business Plan Competition

Three Brown teams, all with connections to the School of Engineering, were named finalists at the 2012 Rhode Island Business Plan Competition. In total, 75 emerging entrepreneurs applied to the competition, and seven have been selected as finalists, including four in the entrepreneur track and three in the student track. All three of the finalists in the student track were from Brown and the School of Engineering.

One of the finalists, JCD Wind, was from Steve Petteruti’s Entrepreneurship I class and Entrepreneurship II classes, Engineering 1930G and Engineering 1930H. JCD Wind, included James McGinn ’12, a biomedical engineering concentrator, and Carli Wiesenfeld ’12, a commerce, organizations, and entrepreneurship (COE) concentrator. The company aims to make seamless, high strength lightweight carbon fiber turbine blades. McGinn is also a member of the men’s rugby team, and Wiesenfeld is a member of the women’s gymnastics team.

A group of four graduate students in the Program in Innovation Management and Entrepreneurship (PRIME) master’s degree program were also among the semifinalists. Solar4Cents included PRIME students Sean Pennino, Bhavuk Nagpal, Xiaotong 'Peter' Shan, and Meng 'Milo' Zhang. Solar4Cents is a manufacturing company that aims to produce low-cost, thin-film copper, zinc, tin, and sulfur solar cells for solar panel manufacturers.

The third finalist in the student track is Overhead.fm, which includes Brown undergraduates Parker Wells '12, a mechanical engineering concentrator, and Stephen Hebson '12, an economics and history double concentrator. Their plan is to produce a web app to provide a customizable stream of music for in-store use, licensing music directly from the artists and labels.

The contest will award more than $200,000 in prizes. In order to receive prizes, applicants must agree to establish or continue operations in Rhode Island. Winners will be announced on May 3 at RIBX 2012 (Business Expo) at the Rhode Island Convention Center in Providence. The event is free and open to the public. All finalists will make brief presentations of their business plans at the Expo. Please go to www.ri-bizplan.comfor more details. Established in 2000, the Rhode Island Business Plan Competition has awarded more than $1 million in prizes since its inception.

For the official RI Business plan release on the competition, please go to:
http://www.ri-bizplan.com/DefaultPermissions/2012FinalistsNamed/tabid/258/Default.aspx

For the Providence Business News story on the event, please go to: http://www.pbn.com/Biz-plan-competition-names-finalists,67143?category_id=27&sub_type=stories,packages

Read More

Wei Yang PhD ‘85 among eight honorary degree recipients at Brown Commencement

During its 244th Commencement, Brown University will confer eight honorary doctorates: Carolyn Bertozzi, biochemist; Viola Davis, actress; John Lewis, civil rights leader; Marilynne Robinson, writer; Sebastian Ruth, musician; Diane Sawyer, journalist; Gene Sharp, political theorist; and Wei Yang, engineer.

PROVIDENCE, R.I. [Brown University] — Eight people who have distinguished themselves through their efforts in the arts, sciences, letters, scholarship and public service will receive honorary degrees from Brown University at Commencement on Sunday, May 27, 2012:

·         Carolyn Bertozzi, chemist and immunologist;

·         Viola Davis, actress;

·         John Lewis, civil rights leader and U.S. representative;

·         Marilynne Robinson, writer and educator;

·         Sebastian Ruth, musician and educator;

·         Diane Sawyer, broadcast journalist;

·         Gene Sharp, political theorist, scholar of nonviolent change; and

·         Wei Yang, engineer.

Honorary degrees are awarded by the University’s Board of Fellows and are conferred by the University president — in English and in Latin — during Commencement exercises on the College Green.

None of the recipients will speak at the Commencement ceremony; that honor has been reserved since the University’s earliest days for two members of the graduating class. Several of the honorands will, however, participate in Commencement forums and other public presentations during Commencement Weekend. Information about times and places for these and other Commencement presentations will be available from the Office of Media Relations and on the University’s 2012 Commencement webpage.

 

Wei Yang  
Engineer and President of Zhejiang University
Doctor of Science (Sc.D.)

Wei Yang, president of China’s Zhejiang University, is an internationally celebrated engineer and materials science researcher, educator, and administrator. He is a Ph.D. graduate of Brown University.

Born in Beijing, Yang was educated in the United States and China (B.S., Northwestern Polytechnic University, 1976; M.S., Tsinghua University, 1981; Ph.D., Brown University, 1985). Four years after earning his Ph.D., Yang was promoted to a full professor of engineering at Tsinghua University, the youngest person ever to achieve that rank.

In addition to continuing an active and very productive career as a research engineer in fracture mechanics, mechatronic reliability, and micro/nanomechanics (11 books and 211 technical papers in internationally refereed journals), Yang has served in a number of national and international positions as an educator and administrator. He became director of the Failure Mechanics Laboratory of the Chinese Ministry of Education in 1993. For seven years (1997–2004), he headed the Department of Engineering Mechanics at Tsinghua, also serving for a time as executive dean of the Aerospace School. From 2004 to 2006, he served as director-general of the Academic Degrees Committee of the State Council of China and also headed the Directorate of Graduate Education. He began as president of Zhejiang University, one of China’s largest and oldest universities, in 2006.

As head of the Chinese Academy of Sciences Technological Science Division, Yang has had extensive international scientific experience. He has served as regional editor for several journals in the field of composite materials and has been on the editorial boards of the International Journal of Fracture, Fatigue & Fracture of Engineering Materials & Structures, and the Archive of Applied Mechanics, among several others. Early in 2009, Yang became widely known for a stern and dedicated fight against scientific misconduct, dealing strictly with researchers found to have engaged in misconduct and proactively developing training programs to support scientific integrity. Organizations and journals,Nature among them, praised his zero tolerance policies.

Yang has supported and worked for collaborations with universities in the United States, Germany, the Netherlands, Singapore and elsewhere. His own postgraduate students — more than 40 of them — have extended his international reach. More than 10 of them hold engineering faculty positions in the United States and Europe. He has been honored extensively for his efforts, including the 2009 Brown University Engineering Alumni Medal.

Read More

Posted in yang | No comments

Five Brown Teams Named Semifinalists in Rhode Island Business Plan Competition

Five Brown teams, including three with connections to the School of Engineering, were named semifinalists at the 2012 Rhode Island Business Plan Competition. In total, 75 emerging entrepreneurs applied to the competition, and 13 were selected as semifinalists, including seven in the entrepreneur track and six in the student track. Five of the six semifinalists in the student track were from Brown, including three teams from the School of Engineering.

Two teams from Steve Petteruti’s Entrepreneurship I class and Entrepreneurship II classes, Engineering 1930G and Engineering 1930H, were among the semifinalists. JCD Wind, included James McGinn ’12, a biomedical engineering concentrator, and Carli Wiesenfeld ’12, a commerce, organizations, and entrepreneurship (COE) concentrator. The company aims to make seamless, high strength lightweight carbon fiber turbine blades. McGinn is also a member of the men’s rugby team, and Wiesenfeld is a member of the women’s gymnastics team.

Another semifinalist was CityBase Security, which included Julianne Bishop ’12 and Joschka Tryba ’12, who are both COE concentrators. Their company hopes to provide a map-based platform that will integrate communication and information systems for both emergency responders and public reporting of incidents/crimes.

A group of four graduate students in the Program in Innovation Management and Entrepreneurship (PRIME) master’s degree program were also among the semifinalists. Solar4Cents included PRIME students Sean Pennino, Bhavuk Nagpal, Xiaotong 'Peter' Shan, and Meng 'Milo' Zhang. Solar4Cents is a manufacturing company that aims to produce low-cost, thin-film copper, zinc, tin, and sulfur solar cells for solar panel manufacturers.

The contest will award more than $200,000 in prizes. In order to receive prizes, applicants must agree to establish or continue operations in Rhode Island. Winners will be announced on May 3. Please go to www.ri-bizplan.com for more details. Established in 2000, the Rhode Island Business Plan Competition has awarded more than $1 million in prizes since its inception.

For the official RI Business plan release on the competition, please go to: http://www.ri-bizplan.com/DefaultPermissions/SemifinalistsNamed/tabid/257/Default.aspx

For the Providence Business News story on the event, please go to: http://www.pbn.com/Biz-plan-competition-names-semi-finalists,66767?category_id=27&sub_type=stories,packages

For the GoLocalProv.com story, please go to: http://www.golocalprov.com/business/new-ri-business-plan-competition-2012-names-semi-finalists/

Read More

Bats save energy by drawing in wings on upstroke

Bat wings are like hands: meaty, bony and full of joints. A new Brown University study finds that bats take advantage of their flexibility by folding in their wings on the upstroke to save inertial energy. The research suggests that engineers looking at flapping flight should account for wing mass and consider a folding design.

PROVIDENCE, R.I. [Brown University] — Whether people are building a flying machine or nature is evolving one, there is pressure to optimize efficiency. A new analysis by biologists, physicists, and engineers at Brown University reveals the subtle but important degree to which that pressure has literally shaped the flapping wings of bats.

The team’s observations and calculations show that by flexing their wings inward to their bodies on the upstroke, bats use only 65 percent of the inertial energy they would expend if they kept their wings fully outstretched. Unlike insects, bats have heavy, muscular wings with hand-like bendable joints. The study suggests that they use their flexibility to compensate for that mass.

“Wing mass is important and it’s normally not considered in flight,” said Attila Bergou, who along with Daniel Riskin is co-lead author of the study that appears April 11 in theProceedings of the Royal Society B. “Typically you analyze lift, drag, and you don’t talk about the energy of moving the wings.”

The findings not only help explain why bats and some birds tuck in their wings on the upstroke, but could also help inform human designers of small flapping vehicles. The team’s research is funded by the U.S. Air Force Office of Sponsored Research.

“If you have a vehicle that has heavy wings, it would become energetically beneficial to fold the wings on the upstroke,” said Sharon Swartz, professor of ecology and evolutionary biology at Brown. She and Kenneth Breuer, professor of engineering, are senior authors on the paper.

The physics of flexed flapping
The team originally set out to study something different: how wing motions vary among bats along a wide continuum of sizes. They published those results in 2010 in the Journal of Experimental Biology, but as they analyzed the data further, they started to consider the intriguing pattern of the inward flex on the upstroke.

That curiosity gave them a new perspective on their 1,000 frames-per-second videos of 27 bats performing five trials each aloft in a flight corridor or wind tunnel. They tracked markers on the bats, who hailed from six species, and measured how frequently the wings flapped, how far up and down they flapped, and the distribution of mass within them as they moved. They measured the mass by cutting the wing of a bat that had died into 32 pieces and weighing them.

The team fed the data in to a calculus-rich model that allowed them to determine what the inertial energy costs of flapping were and what they would have been if the wings were kept outstretched.

Bergou, a physicisist, said he was surprised that the energy savings was so great, especially because the calculations also showed that the bats have to spend a lot of energy — 44 percent of the total inertial cost of flapping — to fold their wings inward and then back outward ahead of the downstroke.

“Retracting your wings has an inertial cost,” Bergou said. “It is significant but it is outweighed by the savings on the up and down stroke.”

The conventional wisdom has always been that bats drew their wings in on the upstroke to reduce drag in the air, and although the team did not measure that, they acknowledge that aerodynamics plays the bigger role in the overall energy budget of flying. But the newly measured inertial savings of drawing in the wings on the upstroke seems too significant to be an accident.

“It really is an open question whether natural selection is so intense on the design and movement patterns of bats that it reaches details of how bats fold their wings,” Swartz said. “This certainly suggests that this is not a random movement pattern and that it is likely that there is an energetic benefit to animals doing this.”

- by David Orenstein

Read More