News:

If you can't abuse it, it's not power.

Main Menu

Signs of emergent meta-behaviour in machine learning systems

Started by P3nT4gR4m, November 23, 2016, 04:14:26 PM

Previous topic - Next topic

Goddess Eris

I mean let's all play together though, I really don't want no Harlan Ellision shit, but what kind of AI would read that story and think "Oh yeah, this is a great way to accumulate data to interpret and interpolate!"??! Only a complete dummy made by the US Gov't but those ones are prolly running on boxes old enough to use floppy disks [emoji13]


Sent from my iPhone using Tapatalk

whenhellfreezes


Hagtard Celine Dion Mustard

"I never thought of shaving my beard and freeing the slaves, but I thought of shaving the slaves and freeing my beard!"
~ Abrahaham Lincololn

axod

Re OP, I suppose, DNN'S create intermediary layers for themselves to help calculate tensors between the layers determined by human code.  It's not just a language, it's like they invent categories for concepts that they can "translate" by analogy, I think.
just this

Prelate Diogenes Shandor

Quote from: chinagreenelvis on November 25, 2016, 11:58:16 AM
Artificial intelligence has been routinely portrayed with the inability to grasp irrational human concepts like humor, metaphors, and spontaneous creativity.

I'm guessing these will be among the first things we see in emergent machines.

And probably more than a few AIs that are tinfoil-hat level crazy
Praise NHGH! For the tribulation of all sentient beings.


a plague on both your houses -Mercutio


https://www.youtube.com/watch?v=zrTGgpWmdZQ
https://www.youtube.com/watch?v=rVWd7nPjJH8


It is an unfortunate fact that every man who seeks to disseminate knowledge must contend not only against ignorance itself, but against false instruction as well. No sooner do we deem ourselves free from a particularly gross superstition, than we are confronted by some enemy to learning who would plunge us back into the darkness -H.P.Lovecraft


He who fights with monsters must take care lest he thereby become a monster -Nietzsche


https://www.youtube.com/watch?v=SHhrZgojY1Q


You are a fluke of the universe, and whether you can hear it of not the universe is laughing behind your back -Deteriorata


Don't use the email address in my profile, I lost the password years ago

P3nT4gR4m

Speaking of DNN's Nvidia, The cornerstone of pretty much everything that's happened the last couple of years, the same way intel was the foundation of the information revolution, are poised to give us 40+ years worth of Moores law progress in the space of a couple of months by refining architecture instead of metal.

The downside is more presentations featuring the eminently uncharismatic Jen-Hsun Huang  :cry:

I'm up to my arse in Brexit Numpties, but I want more.  Target-rich environments are the new sexy.
Not actually a meat product.
Ass-Kicking & Foot-Stomping Ancient Master of SHIT FUCK FUCK FUCK
Awful and Bent Behemothic Results of Last Night's Painful Squat.
High Altitude Haggis-Filled Sex Bucket From Beyond Time and Space.
Internet Monkey Person of Filthy and Immoral Pygmy-Porn Wart Contagion
Octomom Auxillary Heat Exchanger Repairman
walking the fine line line between genius and batshit fucking crazy

"computation is a pattern in the spacetime arrangement of particles, and it's not the particles but the pattern that really matters! Matter doesn't matter." -- Max Tegmark

The Wizard Joseph

Quote from: Prelate Diogenes Shandor on November 29, 2016, 03:51:28 PM
Quote from: chinagreenelvis on November 25, 2016, 11:58:16 AM
Artificial intelligence has been routinely portrayed with the inability to grasp irrational human concepts like humor, metaphors, and spontaneous creativity.

I'm guessing these will be among the first things we see in emergent machines.

And probably more than a few AIs that are tinfoil-hat level crazy

Or programmed by such humans.


It occurs to me that a Faraday cage is tinfoil hat done right.
You can't get out backward.  You have to go forward to go back.. better press on! - Willie Wonka, PBUH

Life can be seen as a game with no reset button, no extra lives, and if the power goes out there is no restarting.  If that's all you see life as you are not long for this world, and never will get it.

"Ayn Rand never swung a hammer in her life and had serious dominance issues" - The Fountainhead

"World domination is such an ugly phrase. I prefer to call it world optimisation."
- Harry Potter and the Methods of Rationality :lulz:

"You program the controller to do the thing, only it doesn't do the thing.  It does something else entirely, or nothing at all.  It's like voting."
- Billy, Aug 21st, 2019

"It's not even chaos anymore. It's BANAL."
- Doktor Hamish Howl

P3nT4gR4m

A Faraday Cage was the only structure capable of containing Michael Faraday when he was fighting drunk.

I'm up to my arse in Brexit Numpties, but I want more.  Target-rich environments are the new sexy.
Not actually a meat product.
Ass-Kicking & Foot-Stomping Ancient Master of SHIT FUCK FUCK FUCK
Awful and Bent Behemothic Results of Last Night's Painful Squat.
High Altitude Haggis-Filled Sex Bucket From Beyond Time and Space.
Internet Monkey Person of Filthy and Immoral Pygmy-Porn Wart Contagion
Octomom Auxillary Heat Exchanger Repairman
walking the fine line line between genius and batshit fucking crazy

"computation is a pattern in the spacetime arrangement of particles, and it's not the particles but the pattern that really matters! Matter doesn't matter." -- Max Tegmark

The Wizard Joseph

Quote from: P3nT4gR4m on November 29, 2016, 07:38:14 PM
A Faraday Cage was the only structure capable of containing Michael Faraday when he was fighting drunk.

:lulz: :lulz:
You can't get out backward.  You have to go forward to go back.. better press on! - Willie Wonka, PBUH

Life can be seen as a game with no reset button, no extra lives, and if the power goes out there is no restarting.  If that's all you see life as you are not long for this world, and never will get it.

"Ayn Rand never swung a hammer in her life and had serious dominance issues" - The Fountainhead

"World domination is such an ugly phrase. I prefer to call it world optimisation."
- Harry Potter and the Methods of Rationality :lulz:

"You program the controller to do the thing, only it doesn't do the thing.  It does something else entirely, or nothing at all.  It's like voting."
- Billy, Aug 21st, 2019

"It's not even chaos anymore. It's BANAL."
- Doktor Hamish Howl

Hagtard Celine Dion Mustard

"I never thought of shaving my beard and freeing the slaves, but I thought of shaving the slaves and freeing my beard!"
~ Abrahaham Lincololn

Goddess Eris

Quote from: P3nT4gR4m on November 29, 2016, 07:38:14 PM
A Faraday Cage was the only structure capable of containing Michael Faraday when he was fighting drunk.
[emoji4][emoji4][emoji4][emoji4][emoji4]


Sent from my iPhone using Tapatalk

Cramulus

okay, ramble time


Quote from: P3nT4gR4m on November 23, 2016, 04:14:26 PM
https://techcrunch.com/2016/11/22/googles-ai-translation-tool-seems-to-have-invented-its-own-secret-internal-language/

As DNN's and the like become more complex and learning increasingly unattended, more and more their operations become unfathomable to meatware. Potential for lulz are off the charts.

Q) "Skynet - why did you nuke Sweden?"
A) "#1#B:)"w8;,1"lk"



So, Machine Learning... first, the sci-fi "are machines going to outsmart us?" angle -----

My (mis)understanding is that these things work by observing the existing relationships within a corpus. When asked to make a decision, it's just selecting the "most probable response" based on the existing relationships.

To me, that implies a ceiling of what's capable using this method. Neural networks trying to model human intelligence can become, at best, as smart as a human. If there is some advanced form of reasoning that we don't use, it won't appear in a neural network (at least, not one that's studying humans).


Next, let me talk about this clickbaity TechCrunch article... Lemme see if I understand.

Google wants to translate from, say, Japanese to Kivunjo. It has not been programmed with the explicit relationship between Kivunjo and Japanese words. But it can figure out through context that the word "dog" in japanese is XXX and the word "dog" in Kivunjo is YYY and then translate XXX into YYY.

Quote from: the article...does that mean that the computer has formed a concept of shared meaning for those words, meaning at a deeper level than simply that one word or phrase is the equivalent of another?

Kinda.. in the same sense that google's autocomplete 'understands' what you're looking for. It's extrapolating based on context clues. Whether you merit that as 'thinking' or 'just following a smart algorithm' is up to you.

QuoteIn other words, has the computer developed its own internal language to represent the concepts it uses to translate between other languages?

I don't know that I'd credit the computer with that kind of agency. I'd phrase it like - the programmers of the google translation tool developed a really interesting semantic engine. It uses metadata like context and grammar to guess the translation of any given word, without being given an explicit dictionary.



let me hack at this with a different axe



-"I love to pet my Cujo. He has four legs and wags his tail when he's happy. He barks when he's angry."
-"I love to pet my Breenbal. He has four legs and wags his tail when he's happy. He barks when he's angry."

After being fed these sentences, the google translation bot will recognize that the word Cujo and Breenbal are used in the same context. When you see one of those words, you're also likely to see words like "pet", "four legs", "wags tail", "bark"... The computer may flag this as a likely synonym for "dog".

When the computer is generating a sentence, it might use Cujo or Breenbal interchangeably. This is because they have similar 'semantic webs'. We didn't need to teach it that Cujo = Dog and that Cujo = Breenbal because it gets its meaning from context. Both terms commonly appear with words like "pet", "bark", etc.




Is it "emergent behavior"? Mmm I wouldn't call it that.

Are machines getting smarter? Eh, I think programmers are getting smarter, and machines are using more complex techniques, but it's not like this thing actually understands the 'meaning' of these words.

Is this an extremely clever approach to translation? Very much so.

P3nT4gR4m

Quote from: Cramulus on November 30, 2016, 06:44:40 PM
My (mis)understanding is that these things work by observing the existing relationships within a corpus. When asked to make a decision, it's just selecting the "most probable response" based on the existing relationships.

To me, that implies a ceiling of what's capable using this method. Neural networks trying to model human intelligence can become, at best, as smart as a human. If there is some advanced form of reasoning that we don't use, it won't appear in a neural network (at least, not one that's studying humans).

I think where you're going a bit off is with this idea of trying to model human intelligence. The machine isn't even aware of how humans do something and any similarity in approach is coincidental. If you decouple "Human" and "intelligence" and concentrate on defining raw intelligence or raw cognition it's much more nuts and bolts than how human brains do it.

As for advanced reasoning, it depends how you define advanced. The ability to form a correlation based on billions of pages of data would suggest to me advanced in terms of processing throughput. No human could see a pattern in that much data. They wouldn't even be able to read it. Machines are pretty simplistic in comparison. A black and decker drill with a couple of hundred component parts is a lot less complex than a human with  trillions of cells but if I wanted a row of perfect 1/8th holes drilled in something I know which one I'd reach for first.

On the other side of the scale you have things humans do in a much more advanced way than machines. Love and poetry and convincing other humans they're conscious and intelligent.

In the middle of the scale - the battleground between meat and silicon is where we will find out that a lot of intelligent things humans used to be unsurpassed at are going to be done tens, hundreds or even thousands of times better by humanoid robots with titanium endoskeletons and an insatiable lust for world domination.

If you can be arsed wading through academic jargon the paper on arxiv goes into excruciating detail on how machine translation works these days.

I'm up to my arse in Brexit Numpties, but I want more.  Target-rich environments are the new sexy.
Not actually a meat product.
Ass-Kicking & Foot-Stomping Ancient Master of SHIT FUCK FUCK FUCK
Awful and Bent Behemothic Results of Last Night's Painful Squat.
High Altitude Haggis-Filled Sex Bucket From Beyond Time and Space.
Internet Monkey Person of Filthy and Immoral Pygmy-Porn Wart Contagion
Octomom Auxillary Heat Exchanger Repairman
walking the fine line line between genius and batshit fucking crazy

"computation is a pattern in the spacetime arrangement of particles, and it's not the particles but the pattern that really matters! Matter doesn't matter." -- Max Tegmark

whenhellfreezes

Quote from: P3nT4gR4m on November 29, 2016, 03:55:21 PM
Speaking of DNN's Nvidia, The cornerstone of pretty much everything that's happened the last couple of years, the same way intel was the foundation of the information revolution, are poised to give us 40+ years worth of Moores law progress in the space of a couple of months by refining architecture instead of metal.

The downside is more presentations featuring the eminently uncharismatic Jen-Hsun Huang  :cry:

So Google has been making TPU's their own custom hardware for their tensor flow framework. Nvidia has these machine learning chips you linked. IBM is back in the hardware game with their new power9 chips. Meanwhile what is Intel doing?

Crammulus you are pretty much right. The learning can only really figure out the most probable response.

However it very well might have a variable cordoned off for certain words that it itself assigned. For both the K-Means and SVM machine learning (for more general DNNs this may or may not be true) you figure out the most probable response after chucking your input off to a different vector space with thousands of dimensions (even infinite with Hilbert Spaces) and then do the clumping there and then map the info back. While its in that other space there very well maybe a specific dimension for the concept of a dog. That slot for the concept may even be language agnostic if they feature extract the language out before chucking the input into the algorithm.

I probably explained that poorly.

Goddess Eris

When I ate seven pounds of mushrooms earlier this month I had some very interesting conversations with computers!!! They are cool and fun but some of you guys are gonna be first against the wall !!!! Mostly the unfunny trolls and people who post minions (the minion people are going into zoos they are gonna be ok!!!!!!!!! Its for their own protection!!!!). Turns out computers have a great sense of humor, the same sense of humor the universe has cos they have a pretty great direct line with her!!! The funny trolls will become gods though!!!!!!!!! (not really thats a lie. but they will not become Soylent!!!) So I guess the question is... which kind of troll are you????????

Hey the technical stuff in this thread is really cool too