Mediaboss Marketing

Geoffrey Hinton, center. talks about what future deep learning neural nets may look like, flanked by Yann LeCun of Facebook, left, and Yoshua Bengio of Montreal's MILA institute for AI, during a press conference at the 34th annual AAAI conference on artificial intelligence.

The rise of dedicated chips and systems for artificial intelligence will "make possible a lot of stuff that's not possible now," said Geoffrey Hinton, the University of Toronto professor who is one of the godfathers of the "deep learning" school of artificial intelligence, during a press conference on Monday.

Hinton joined his compatriots, Yann LeCun of Facebook and Yoshua Bengio of Canada's MILA institute, fellow deep learning pioneers, in an upstairs meeting room of the Hilton Hotel on the sidelines of the 34th annual conference on AI by the Association for the Advancement of Artificial Intelligence. They spoke for 45 minutes to a small group of reporters on a variety of topics, including AI ethics and what "common sense" might mean in AI. The night before, all three had presented their latest research directions.

Regarding hardware, Hinton went into an extended explanation of the technical aspects that constrain today's neural networks. The weights of a neural network, for example, have to be used hundreds of times, he pointed out, making frequent, temporary updates to the weights. He said the fact graphics processing units (GPUs) have limited memory for weights and have to constantly store and retrieve them in external DRAM is a limiting factor.

Much larger on-chip memory capacity "will help with things like Transformer, for soft attention," said Hinton, referring to the wildly popular autoregressive neural network developed at Google in 2017. Transformers, which use "key/value" pairs to store and retrieve from memory, could be much larger with a chip that has substantial embedded memory, he said.

Also: Deep learning godfathers Bengio, Hinton, and LeCun say the field can fix its flaws

LeCun and Bengio agreed, with LeCun noting that GPUs "force us to do batching," where data samples are combined in groups as they pass through a neural network, "which isn't efficient." Another problem is that GPUs assume neural networks are built out of matrix products, which forces constraints on the kind of transformations scientists can build into such networks.

"Also sparse computation, which isn't convenient to run on GPUs ...," said Bengio, referring to instances where most of the data, such as pixel values, may be empty, with only a few significant bits to work on.

LeCun predicted that new hardware would lead to "much bigger neural nets with sparse activations," and he and Bengio both emphasized that there is an interest in doing the same amount of work with less energy. LeCun defended AI against claims it is an energy hog, however. "This idea that AI is eating the atmosphere, it's just wrong," he said. "I mean, just compare it to something like raising cows," he continued. "The energy consumed by Facebook annually for each Facebook user is 1,500-watt hours," he said. Not a lot, in his view, compared to other energy-hogging technologies.

The biggest problem with hardware, mused LeCun, is that on the training side of things, it is a duopoly between Nvidia, for GPUs, and Google's Tensor Processing Unit (TPU), repeating a point he had made last year at the International Solid-State Circuits Conference.

Even more interesting than hardware for training, LeCun said, is hardware design for inference. "You now want to run on an augmented reality device, say, and you need a chip that consumes milliwatts of power and runs for an entire day on a battery." LeCun reiterated a statement made a year ago that Facebook is working on various internal hardware projects for AI, including for inference, but he declined to go into details.

Also: Facebook's Yann LeCun says 'internal activity' proceeds on AI chips

Today's neural networks are tiny, Hinton noted, with really big ones having perhaps just ten billion parameters. Progress on hardware might advance AI just by making much bigger nets with an order of magnitude more weights. "There are one trillion synapses in a cubic centimeter of the brain," he noted. "If there is such a thing as General AI, it would probably require one trillion synapses."

As for what "common sense" might look like in a machine, nobody really knows, Bengio maintained. Hinton complained people keep moving the goalposts, such as with natural language models. "We finally did it, and then they said it's not really understanding, and can you figure out the pronoun references in the Winograd Schema Challenge," a question-answering task used a computer language benchmark. "Now we are doing pretty well at that, and they want to find something else" to judge machine learning he said. "It's like trying to argue with a religious person, there's no way you can win."

But, one reporter asked, what's concerning to the public is not so much the lack of evidence of human understanding, but evidence that machines are operating in alien ways, such as the "adversarial examples." Hinton replied that adversarial examples show the behavior of classifiers is not quite right yet. "Although we are able to classify things correctly, the networks are doing it absolutely for the wrong reasons," he said. "Adversarial examples show us that machines are doing things in ways that are different from us."

LeCun pointed out animals can also be fooled just like machines. "You can design a test so it would be right for a human, but it wouldn't work for this other creature," he mused. Hinton concurred, observing "house cats have this same limitation."

Also: LeCun, Hinton, Bengio: AI conspirators awarded prestigious Turing prize

"You have a cat lying on a staircase, and if you bounce a soccer ball down the stairs toward a care, the cat will just sort of watch the ball bounce until it hits the cat in the face."

Another thing that could prove a giant advance for AI, all three agreed, is robotics. "We are at the beginning of a revolution," said Hinton. "It's going to be a big deal" to many applications such as vision. Rather than analyzing the entire contents of a static image or video frame, a robot creates a new "model of perception," he said.

"You're going to look somewhere, and then look somewhere else, so it now becomes a sequential process that involves acts of attention," he explained.

Hinton predicted last year's work by OpenAI in manipulating a Rubik's cube was a watershed moment for robotics, or, rather, an "AlphaGo moment," as he put it, referring to DeepMind's Go computer.

LeCun concurred, saying that Facebook is running AI projects not because Facebook has an extreme interest in robotics, per se, but because it is seen as an "important substrate for advances in AI research."

It wasn't all gee-whiz, the three scientists offered skepticism on some points. While most research in deep learning that matters is done out in the open, some companies boast of AI while keeping the details a secret.

"It's hidden because it's making it seem important," said Bengio, when in fact, a lot of work in the depths of companies may not be groundbreaking. "Sometimes companies make it look a lot more sophisticated than it is."

Bengio continued his role among the three of being much more outspoken on societal issues of AI, such as building ethical systems.

When LeCun was asked about the use of facial recognition algorithms, he noted technology can be used for good and bad purposes, and that a lot depends on the democratic institutions of society. But Bengio pushed back slightly, saying, "What Yann is saying is clearly true, but prominent scientists have a responsibility to speak out." LeCun mused that it's not the job of science to "decide for society," prompting Bengio to respond, "I'm not saying decide, I'm saying we should weigh in because governments in some countries are open to that involvement."

Hinton, who frequently punctuates things with a humorous aside, noted toward the end of the gathering his biggest mistake with respect to Nvidia. "I made a big mistake back in 2009 with Nvidia," he said. "In 2009, I told an audience of 1,000 grad students they should go and buy Nvidia GPUs to speed up their neural nets. I called Nvidia and said I just recommended your GPUs to 1,000 researchers, can you give me a free one, and they said, No.

"What I should have done, if I was really smart, was take all my savings and put it into Nvidia stock. The stock was at $20 then, now it's, like, $250."

View original post here:
AI on steroids: Much bigger neural nets to come with new hardware, say Bengio, Hinton, and LeCun - ZDNet

AI has fallen from glorious summers into dismal winters before. The temptation to predict another such tumble recurs naturally. So that is the question the BBC posed to AI researchers: Are we on the cusp of an AI winter:

The 10s were arguably the hottest AI summer on record with tech giants repeatedly touting AIs abilities.

AI pioneer Yoshua Bengio, sometimes called one of the godfathers of AI, told the BBC that AIs abilities were somewhat overhyped in the 10s by certain companies with an interest in doing so.

There are signs, however, that the hype might be about to start cooling off.

I keep up with this kind of thing. The answer is: Yes, and no. AI did surge past milestones during the 2010s that it had not been expected to cross for many more years:

2011 IBMs Watson wins at Jeopardy! IBM Watson: The inside story of how the Jeopardy-winning supercomputer was born, and what it wants to do next (Tech Republic, September 9, 2013)

2012 Google unveils a deep learning systems that recognized images of cats

2015 Image recognition systems outperformed humans in the ImageNet challenge

2016 AlphaGo defeats world Go champion Lee Sedol: In Two Moves, AlphaGo and Lee Sedol Redefined the Future (Wired, March 16, 2016)

2018 Self-driving cars hit the road as Googles Waymo launched (a very limited) self-driving taxi service in Phoenix, Arizona

But other headlines during the period have been less heeded:

Despite High Hopes, Self-Driving Cars Are Way in the Future (2019)

The Next Hot Job: Pretending to Be a Robot (2019)

Boeings Sidelined Fuselage Robots: What Went Wrong? (2019)

Self-driving cars: Hype-filled decade ends on sobering note (2019)

Tesla driver killed in crash with Autopilot active, NHTSA investigating (2016)

Dont fall for these 3 myths about AI, machine learning (2018)

A Sobering Message About the Future at AIs Biggest Party (2019)

And so on.

So which is it? AI Winter or Robot Overlords? I suggest neither. And so do active researchers.

Gary Marcus, an AI researcher at New York University, said: By the end of the decade there was a growing realisation that current techniques can only carry us so far.

He thinks the industry needs some real innovation to go further.

There is a general feeling of plateau, said Verena Rieser, a professor in conversational AI at Edinburgh[s Heriot Watt University.

One AI researcher who wishes to remain anonymous said were entering a period where we are especially sceptical about AGI.

Recent AI developments, notably those lumped under the rubric of Deep Learning have advanced the state-of-the-art in machine learning. Lets not forget that prior efforts, such as the poorly named Expert Systems, had faded because, well, they werent expert at all. Deep Learning systems, as highly flexible pattern matchers, will endure.

What is not coming is the long-predicted AI Overlord, or anything that is even close to surpassing human intelligence. Like any other tool we build, AI has its place when it amplifies and augments our abilities.

Just as tractors and diggers have not led to legions of people who no longer use their arms, the latest advances in AI will not lead to human serfs cowering before beneath an all-intelligent machine. If anything, AI will require more from us, not less, because how we choose to use these tools will make an increasingly stark difference between benefit and ruin.

As Samin Winiger, a former AI research at Google says, What we called AI or machine learning during the past 10-20 years, will be seen as just yet another form of computation

Machines are tool in the toolbox, not a replacement for minds. An AI winter would only be coming if we forgot that.

Here are some of Brendan Dixons earlier musings on the concept of an AI Winter:

Just a light frost? Or an AI winter? Its nice to be right once in a whilecheck out the evidence for yourself

and

AI WinterIs Coming:Roughly every decade since the late 1960s has experienced a promising wave of AI that later crashed on real-world problems, leading to collapses in research funding.

Read more here:
So Is an AI Winter Really Coming This Time? - Walter Bradley Center for Natural and Artificial Intelligence

A team of researchers from Aarhus University in Denmark let DeepMinds AlphaZero algorithm loose on a few quantum computing optimization problems and, much to everyones surprise, the AI was able to solve the problems without any outside expert knowledge. Not bad for a machine learning paradigm designed to win at games like Chess and StarCraft.

Youve probably heard of DeepMind and its AI systems. The UK-based Google sister-company is responsible for both AlphaZero and AlphaGo, the systems that beat the worlds most skilled humans at the games of Chess and Go. In essence, what both systems do is try to figure out what the optimal next set of moves is. Where humans can only think so many moves ahead, the AI can look a bit further using optimized search and planning methods.

Related:DeepMinds AlphaZero AI is the new champion in chess, shogi, and Go

When the Aarhus team applied AlphaZeros optimization abilities to a trio of problems associated with optimizing quantum functions an open problem for the quantum computing world they learned that its ability to learn new parameters unsupervised transferred over from games to applications quite well.

Per the study:

AlphaZero employs a deep neural network in conjunction with deep lookahead in a guided tree search, which allows for predictive hidden-variable approximation of the quantum parameter landscape. To emphasize transferability, we apply and benchmark the algorithm on three classes of control problems using only a single common set of algorithmic hyperparameters.

The implications for AlphaZeros mastery over the quantum universe could be huge. Controlling a quantum computer requires an AI solution because operations at the quantum level quickly become incalculable by humans. The AI can find optimum paths between data clusters in order to emerge better solutions in tandem with computer processors. It works a lot like human heuristics, just scaled to the nth degree.

An example of this would be an algorithm that helps a quantum computer sort through near-infinite combinations of molecules to come up with chemical compounds that would be useful in the treatment of certain illnesses. The current paradigm would involve developing an algorithm that relies on human expertise and databases with previous findings to point it in the right direction.

But the kind of problems were looking at quantum computers to solve dont always have a good starting point. Some of these, optimization problems like the Traveling Salesman Problem, need an algorithm thats capable of figuring things out without the need for constant adjustment by developers.

DeepMinds algorithm and AI system may be the solution quantum computings been waiting for. The researchers effectively employ AlphaZero as a Tabula Rasa for quantum optimization: It doesnt necessarily need human expertise to find the optimum solution to a problem at the quantum computing level.

Before we start getting too concerned about unsupervised AI accessing quantum computers, its worth mentioning that so far AlphaZeros just solved a few problems in order to prove a concept. We know the algorithms can handle quantum optimization, now its time to figure out what we can do with it.

The researchers have already received interest from big tech and other academic institutions with queries related to collaborating on future research. Not for nothing, but DeepMinds sister-company Google has a little quantum computing program of its own. Were betting this isnt the last weve heard of AlphaZeros adventures in the quantum computing world.

Read next: Cyberpunk 2077 has been delayed to September (thank goodness)

Visit link:
AlphaZero beat humans at Chess and StarCraft, now it's working with quantum computers - The Next Web

The Information Age has given rise to many society-shaping tools and technologies, nearly all of them revolving around the gathering, dissemination, and analysis of information. Data collection and data analysis technologies have held a careful balance, but as data collection hardware improves and passive data collection becomes more prevalent, it is not an uncommon problem for transportation agencies to have access to more information than is useful in the scope of human analysis through regular spreadsheet programs. Enter artificial intelligence (AI): the nebulous superhero of data crunching, here to rescue a transportation network inundated with data and make some sense of it all.

With all this data lying around, its important to make the most of it. Agencies have a responsibility to the public to make their networks safe and efficient. This is difficult to do with tight funding and ageing infrastructure, but data is cheap and plentiful - the problem is that the data is spread out across and within agencies, often held in silos, and few people are in a position to take a wide view of the network. The advent of sweeping AI capable of making long-range decisions is inevitable and necessary for the future of the worlds infrastructure. Implementing AI into the planning process is the only way to create an optimal transportation network in the future, and public agencies should be preparing themselves for that eventuality.

Regional and national AIs are inevitable for several reasons. Thanks to fibre networks and 5G, the information of todays world moves at nearly the speed of light. It is unreasonable to expect a human to still make the best possible decisions while overwhelmed with all that information. AI has the power to process the flood of data into human-sized chunks or even make optimal decisions by itself, faster and more consistently than any person.

More importantly than the speed, the true wonder of AI is that it can make better decisions and think on a higher plane than human brains. The DeepMind projects AlphaGo and AlphaGo Zero (see box) are clear examples that - when given the correct scope and enough data - the machines can create strategies that even humans who are masters of their craft cannot understand without retroactive analysis. This same high-level perspective and creativity can be applied to solving transportation problems, and find solutions that the engineers and planners wouldnt have considered.

The primary limitation to all AI is the availability of data, which is what makes transportation planning a perfect area for strategic AI implementation. The transportation industry is data-rich and largely controlled by government organisations everywhere in the world. This makes data collection easier to standardise and provides more rigid boundaries and defined scopes than many other use cases. A unified, holistic view allows planners to design for sustainability, address environmental and social problems, and maximise the available resources even in a flagging economy. As long as it is designed well and phased in properly, strategic regional AIs would give the worlds transportation networks a huge edge when tackling equally huge problems like climate change and suburban sprawl.

There are a few steps agencies should start taking to prepare themselves for the eventuality of strategic AI. The first is the implementation of data-driven processes on fronts that often receive less attention than operations, from project prioritisation to construction and maintenance. Americas public transportation sector has recently been experimenting with a push for data-driven funding and prioritisation decisions. The analysis required for funding applications has been increased, and projects compete through a scoring system based on a number of sustainability metrics which push the overall system further towards sustainable design. The construction side is making a similar shift, with tablet-based inspections, 3D/4D modelling and drones moving to the forefront. All of these serve the goal of creating a system built on data and keeping that data readily available for future use.

Agencies should also focus on connected infrastructure and real-time GIS modelling to expand their passive collection capabilities. Passively gathering data is a vital step in the creation of smart regions and has been the clear direction of transportation networks for some time. Smart city initiatives are common now, but only regional agencies like state departments of transportation have the resources and jurisdiction to start building smart regions. In addition to being part of a transition to AI, these smart regions have added benefits to technologies like connected and autonomous vehicles (C/AVs) and initiatives like Mobility as a Service (MaaS). Taking a wider approach to planning is the best way to start a transition to AI-driven strategy, which means extrapolating ideas that were formerly confined to cities or even city blocks.

From an even broader perspective, public agencies around the world need to start taking a more holistic approach to mobility. The separation of road and bridge, transit, toll, and other transportation entities, is no longer an effective situation. These organisations frequently compete with each other, bickering over right of way and funding, standards and policy, and causing waste on every level of decision-making. Even if strategic AI was not inevitable, breaking down the walls between transportation agencies would still be a necessity. The public sector needs to share data, incorporate first/last mile and micromobility into the networks, and start truly looking at better mobility as a collective goal.

The end goal of all this transition and change is to implement regional strategic AIs into the transportation sector to augment the planning process. These could begin by simply making suggestions, providing data and analysis on demand, and processing a real-time model of the network using all the data the agency has available from both passive and active collection. And who knows what the future of these AIs will bring? Like the AlphaGo programs, they may come up with strategies that human designers would never have considered. They could operate on a higher level of understanding, like a child playing chess with an adult, and we may just have to trust in decisions we dont understand. But with enormous, global problems like climate change around the corner, strategic AI may be the only enormous, global solution we have.

AlphaGo & AlphaGo Zero

Ancient Chinese board game Go is known as the most challenging classical game for artificial intelligence because of its complexity. It involves multiple layers of strategic thinking as players place their own black or white stones on a board to surround and capture their opponents. While this seems relatively simple, there are 10 to the power of 170 possible board configurations - more than the number of atoms in the known universe which makes Go a googol times more complex than chess.

AlphaGo is the first computer program to defeat a professional human Go player, the first to defeat a Go world champion, and is arguably the strongest Go player in history. It combines advanced search tree with deep neural networks, which take a description of the Go board as an input and process it through a number of different network layers containing millions of neuron-like connections.

AlphaGo learnt the game by playing thousands of matches with amateur and professional players. But a more advanced version, AlphaGo Zero, learnt by playing against itself, starting from completely random play and accumulating thousands of years of human knowledge in a few days. It replaced hand-crafted rules with a deep neural network and algorithms that knew nothing beyond the basic rules. Its creative response and ability to master complex games demonstrates that a single algorithm can learn how to discover new knowledge in a range of settings.

Source: DeepMind.com

See more here:
Trust AI it knows more than we do - ITS International

Artificial Intelligence (AI) has made major progress in recent years. But even milestones like AlphaGo or the narrow AI used by big tech only scratch the surface of the seismic changes yet to come.

Modern AI holds the potential to upend entire profession while unleashing brand new industries in the process. Old assumptions will no longer hold, and new realities will dictate those who are swallowed by the tides of change from those able to anticipate and ride the AI wave headlong into a prosperous future.

Heres how businesses and employees can both leverage AI in the 2020s.

Like many emerging technologies, AI comes with a substantial learning curve. As a recent McKinsey report highlights, AI is a slow burn technology that requires a heavy upfront investment, with returns only ramping up well down the road.

Because of this slow burn, an AI front-runner and an AI laggard may initially appear to be on equal footing. The front-runner may even be a bit behind during early growing pains. But as the effects of AI adoption kick in, the gap between the two widens dramatically and exponentially. McKinseys models estimate that within around 10 years, the difference in cumulative net change in cash flow between front-runners and laggards could be as high as 145 percent.

The first lesson for any business hoping to seize new AI opportunities is to start making moves to do so right now.

Read More: How is Artificial Intelligence (AI) Changing the Future of Architecture?

Despite popular opinion, the coming AI wave will be mostly a net positive for employees. The World Economic Forum found that by 2022, AI and Machine Learning will have created over 130 million new jobs. Though impressive, these gains will not be distributed evenly.

Jobs characterized by unskilled and repetitive tasks face an uncertain future, while jobs in need of greater social and creative problem-solving will spike. According to McKinsey, the coming decade could see a 10 percent fall in the share of low digital skill jobs, with a corresponding rise in the share of jobs requiring high digital skill.

So how can employees successfully navigate the coming future of work? One place to start is to investigate the past. Nearly half a century ago, the first ATM was installed outside Barclays Bank in London. In 1967, the thought of bank tellers surviving the introduction of automated teller machines felt impossible. ATMs caught on like wildfire, cut into tellers hours, offered unbeatable flexibility and convenience, and should have all but wiped tellers out.

But, in fact, exactly the opposite happened. No longer having to handle simple deposits freed tellers up to engage with more complex and social facets of the business. They started advising customers on mortgages and loans, forging relationships and winning loyalty. Most remarkable of all, in the years following the ATMs introduction, the total number of tellers employed worldwide didnt fall off a cliff. In fact, it rose higher than ever.

Though AI could potentially threaten some types of jobs, many jobs will see rising demand. Increased reliance on automated systems for core business functions, frees up valuable employee time and enables them to focus on different areas to add even more value to the company.

As employees grow increasingly aware of the changing nature of work, they are also clamoring for avenues for development, aware that they need to hold a variety of skills to remain relevant in a dynamic job market. Companies will, therefore, need to provide employees with a wide range of experiences and the opportunity to continuously enhance their skillsets or suffer high turnover. This is already a vital issue to businesses with the cost of losing an employee equating to 90%-200% of their annual salary. This costs each large enterprise an estimated $400 million a year. If employees feel their role is too restrictive or that their organization is lagging, their likelihood of leaving will climb.

The only way to capture the full value of AI for business is to retain the highly skilled employees capable of wielding it. Departmental silos and rigid job descriptions will have no place in the AI future.

Read More: How Artificial Intelligence and Blockchain is Revolutionizing Mobile Industry in 2020?

For employees to maximize their chances of success in the face of rapid AI advancement, they must remain flexible and continuously acquire new skills. Both businesses and employees will need to realign their priorities in accordance with new realities. Workers will have to be open to novel ideas and perspectives, while employers will need to embrace the latest technological advancements.

Fortunately, the resources and avenues for ambitious employers to pursue continued growth for their employees are blossoming. Indeed, the very AI advancements prompting the need for accelerated career development paths are also powering technologies to maximize and optimize professional enrichment.

AI is truly unlocking an exciting new future of work. Smart algorithms now enable hyper-flexible workplaces to seamlessly shuffle and schedule employee travel, remote work, and mentorship opportunities. At the cutting edge, these technologies can even let employees divide their time between multiple departments across their organization. AI can also tailor training and reskilling programs to each employees unique goals and pace.

The rise of AI holds the promise of great change, but if properly managed, it can be a change for the better.

Read More: Predictions of AI AdTech in 2020

See more here:
Seizing Artificial Intelligence's Opportunities in the 2020s - AiThority