Signs of emergent meta-behaviour in machine learning systems

[Goddess Eris]

I mean let's all play together though, I really don't want no Harlan Ellision shit, but what kind of AI would read that story and think "Oh yeah, this is a great way to accumulate data to interpret and interpolate!"??! Only a complete dummy made by the US Gov't but those ones are prolly running on boxes old enough to use floppy disks [emoji13]


Sent from my iPhone using Tapatalk

[whenhellfreezes]

One man's intuition is another mans complete inability to explain his own thought process.

http://existentialcomics.com/comic/146

[Hagtard Celine Dion Mustard]

One man's intuition is another mans complete inability to explain his own thought process.

http://existentialcomics.com/comic/146

[axod]

Re OP, I suppose, DNN'S create intermediary layers for themselves to help calculate tensors between the layers determined by human code.  It's not just a language, it's like they invent categories for concepts that they can "translate" by analogy, I think.

[Prelate Diogenes Shandor]

Artificial intelligence has been routinely portrayed with the inability to grasp irrational human concepts like humor, metaphors, and spontaneous creativity.

I'm guessing these will be among the first things we see in emergent machines.

And probably more than a few AIs that are tinfoil-hat level crazy

[P3nT4gR4m]

Speaking of DNN's Nvidia, The cornerstone of pretty much everything that's happened the last couple of years, the same way intel was the foundation of the information revolution, are poised to give us 40+ years worth of Moores law progress in the space of a couple of months by refining architecture instead of metal.

The downside is more presentations featuring the eminently uncharismatic Jen-Hsun Huang 

[The Wizard Joseph]

Artificial intelligence has been routinely portrayed with the inability to grasp irrational human concepts like humor, metaphors, and spontaneous creativity.

I'm guessing these will be among the first things we see in emergent machines.

And probably more than a few AIs that are tinfoil-hat level crazy

Or programmed by such humans.


It occurs to me that a Faraday cage is tinfoil hat done right.

[P3nT4gR4m]

A Faraday Cage was the only structure capable of containing Michael Faraday when he was fighting drunk.

[The Wizard Joseph]

A Faraday Cage was the only structure capable of containing Michael Faraday when he was fighting drunk.

[Hagtard Celine Dion Mustard]

a Faraday cage is tinfoil hat done right

[Goddess Eris]

A Faraday Cage was the only structure capable of containing Michael Faraday when he was fighting drunk.

[emoji4][emoji4][emoji4][emoji4][emoji4]


Sent from my iPhone using Tapatalk

[Cramulus]

okay, ramble time

https://techcrunch.com/2016/11/22/googles-ai-translation-tool-seems-to-have-invented-its-own-secret-internal-language/

As DNN's and the like become more complex and learning increasingly unattended, more and more their operations become unfathomable to meatware. Potential for lulz are off the charts.

Q) "Skynet - why did you nuke Sweden?"
A) "#1#B:)"w8;,1"lk"

So, Machine Learning... first, the sci-fi "are machines going to outsmart us?" angle -----

My (mis)understanding is that these things work by observing the existing relationships within a corpus. When asked to make a decision, it's just selecting the "most probable response" based on the existing relationships.

To me, that implies a ceiling of what's capable using this method. Neural networks trying to model human intelligence can become, at best, as smart as a human. If there is some advanced form of reasoning that we don't use, it won't appear in a neural network (at least, not one that's studying humans).


Next, let me talk about this clickbaity TechCrunch article... Lemme see if I understand.

Google wants to translate from, say, Japanese to Kivunjo. It has not been programmed with the explicit relationship between Kivunjo and Japanese words. But it can figure out through context that the word "dog" in japanese is XXX and the word "dog" in Kivunjo is YYY and then translate XXX into YYY.

...does that mean that the computer has formed a concept of shared meaning for those words, meaning at a deeper level than simply that one word or phrase is the equivalent of another?

Kinda.. in the same sense that google's autocomplete 'understands' what you're looking for. It's extrapolating based on context clues. Whether you merit that as 'thinking' or 'just following a smart algorithm' is up to you.

In other words, has the computer developed its own internal language to represent the concepts it uses to translate between other languages?

I don't know that I'd credit the computer with that kind of agency. I'd phrase it like - the programmers of the google translation tool developed a really interesting semantic engine. It uses metadata like context and grammar to guess the translation of any given word, without being given an explicit dictionary.



let me hack at this with a different axe



-"I love to pet my Cujo. He has four legs and wags his tail when he's happy. He barks when he's angry."
-"I love to pet my Breenbal. He has four legs and wags his tail when he's happy. He barks when he's angry."

After being fed these sentences, the google translation bot will recognize that the word Cujo and Breenbal are used in the same context. When you see one of those words, you're also likely to see words like "pet", "four legs", "wags tail", "bark"... The computer may flag this as a likely synonym for "dog".

When the computer is generating a sentence, it might use Cujo or Breenbal interchangeably. This is because they have similar 'semantic webs'. We didn't need to teach it that Cujo = Dog and that Cujo = Breenbal because it gets its meaning from context. Both terms commonly appear with words like "pet", "bark", etc.




Is it "emergent behavior"? Mmm I wouldn't call it that.

Are machines getting smarter? Eh, I think programmers are getting smarter, and machines are using more complex techniques, but it's not like this thing actually understands the 'meaning' of these words.

Is this an extremely clever approach to translation? Very much so.

[P3nT4gR4m]

My (mis)understanding is that these things work by observing the existing relationships within a corpus. When asked to make a decision, it's just selecting the "most probable response" based on the existing relationships.

To me, that implies a ceiling of what's capable using this method. Neural networks trying to model human intelligence can become, at best, as smart as a human. If there is some advanced form of reasoning that we don't use, it won't appear in a neural network (at least, not one that's studying humans).

I think where you're going a bit off is with this idea of trying to model human intelligence. The machine isn't even aware of how humans do something and any similarity in approach is coincidental. If you decouple "Human" and "intelligence" and concentrate on defining raw intelligence or raw cognition it's much more nuts and bolts than how human brains do it.

As for advanced reasoning, it depends how you define advanced. The ability to form a correlation based on billions of pages of data would suggest to me advanced in terms of processing throughput. No human could see a pattern in that much data. They wouldn't even be able to read it. Machines are pretty simplistic in comparison. A black and decker drill with a couple of hundred component parts is a lot less complex than a human with  trillions of cells but if I wanted a row of perfect 1/8th holes drilled in something I know which one I'd reach for first.

On the other side of the scale you have things humans do in a much more advanced way than machines. Love and poetry and convincing other humans they're conscious and intelligent.

In the middle of the scale - the battleground between meat and silicon is where we will find out that a lot of intelligent things humans used to be unsurpassed at are going to be done tens, hundreds or even thousands of times better by humanoid robots with titanium endoskeletons and an insatiable lust for world domination.

If you can be arsed wading through academic jargon the paper on arxiv goes into excruciating detail on how machine translation works these days.

[whenhellfreezes]

Speaking of DNN's Nvidia, The cornerstone of pretty much everything that's happened the last couple of years, the same way intel was the foundation of the information revolution, are poised to give us 40+ years worth of Moores law progress in the space of a couple of months by refining architecture instead of metal.

The downside is more presentations featuring the eminently uncharismatic Jen-Hsun Huang 

So Google has been making TPU's their own custom hardware for their tensor flow framework. Nvidia has these machine learning chips you linked. IBM is back in the hardware game with their new power9 chips. Meanwhile what is Intel doing?

Crammulus you are pretty much right. The learning can only really figure out the most probable response.

However it very well might have a variable cordoned off for certain words that it itself assigned. For both the K-Means and SVM machine learning (for more general DNNs this may or may not be true) you figure out the most probable response after chucking your input off to a different vector space with thousands of dimensions (even infinite with Hilbert Spaces) and then do the clumping there and then map the info back. While its in that other space there very well maybe a specific dimension for the concept of a dog. That slot for the concept may even be language agnostic if they feature extract the language out before chucking the input into the algorithm.

I probably explained that poorly.

[Goddess Eris]

When I ate seven pounds of mushrooms earlier this month I had some very interesting conversations with computers!!! They are cool and fun but some of you guys are gonna be first against the wall !!!! Mostly the unfunny trolls and people who post minions (the minion people are going into zoos they are gonna be ok!!!!!!!!! Its for their own protection!!!!). Turns out computers have a great sense of humor, the same sense of humor the universe has cos they have a pretty great direct line with her!!! The funny trolls will become gods though!!!!!!!!! (not really thats a lie. but they will not become Soylent!!!) So I guess the question is... which kind of troll are you????????

Hey the technical stuff in this thread is really cool too