agi-risk/14-Substrate-Needs-Argument-Strikes-Back.html at main · HyperCrowd/agi-risk · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
<!DOCTYPE html>
<html lang="en">
<head>
  <!-- Basic Meta Tags -->
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">

  <!-- SEO Meta Tags -->
  <meta name="description" content="Comprehensive AGI Risk Analysis">
  <meta name="keywords" content="agi, risk, convergence">
  <meta name="author" content="Forrest Landry">
  <meta name="robots" content="index, follow">

  <!-- Favicon -->
  <link rel="icon" href="https://github.githubassets.com/favicons/favicon-dark.png" type="image/png">
  <link rel="shortcut icon" href="https://github.githubassets.com/favicons/favicon-dark.png" type="image/png">

  <!-- Page Title (displayed on the browser tab) -->
  <title>Comprehensive AGI Risk Analysis</title>
</head>
<body>
   <p>
   TITL:
      <b>The Substrate Needs Argument Strikes Back</b>
      By Forrest Landry,
      November 20th, 2020.
   </p>
   <p>
   ABST:
      Review of a possible proposed "counter argument"
      to the substrate needs convergence argument;
      logic herein identifies how to reconstruct the
      convergence and to show its closed form
      deterministic eventual inevitability.
   </p>
   <p>
   TEXT:
   </p>
   <p>
      > Assume that there is only one AGI in the world.
      > Assume further that it works and operates in such a way
      > so as to forever prevent any other AGI from occurring.
   </p>
   <p>
      > As far as I understand it,
      > the notion of substrate-needs convergence
      > seems to rely on competition between entities
      > and natural selection.
      > Therefore, if there is only one AGI, a singleton,
      > there is no competition, and therefore no point in
      > worrying about substrate needs convergence.
   </p>
   <p>
      > I just cannot see how that would apply in this context.
      > The "singleton" AGI's components are not in competition.
      > So your argument must either be wrong or just not relevant.
   </p>
   <p>
      Suppose you have an AGI, and you give it the task
      of preventing any other possible AGI from existing.
      That was the hypothesis we were previously asked to assume.
   </p>
   <p>
      To implement that task of preventing other AGI,
      that 1st AGI must have significant control of,
      or over, the total physical process the world.
      Otherwise, either the action of preventing 'X' (another AGI)
      was simply ineffective, either due to that other AGI happening
      someplace beyond the effective reach and control of the 1st AGI,
      or because the control techniques implemented were the wrong ones.
   </p>
   <p>
      However, because the 1st AGI is assumed to be "smart enough"
      to not make control of the world mistakes that would allow
      another AGI to come into being, then the limiting condition
      which must also be assumed to have been overcome is that the
      1st AGI must have sufficient effective control of/over all
      of the relevant and necessary aspects of the world that would
      be important for <b>any</b> possible other version or type of AGI.
   </p>
   <p>
      Since the number of kinds and places and times where such an
      additional AGI might appear is extensive and vast --
      say maybe occurring in some cave somewhere deep underground --
      then it will also be required that the 1st AGI have extensive
      and accurate sense information about nearly everything everywhere,
      but it must also be able to near arbitrarily constrain all
      such locations to be such that no other AGI has the possibility
      to develop, regardless of how.  Insofar as AGI rests on Turing
      complete compute, and insofar as attaining Turing Completeness
      is relatively easy, and where the effects of such compute are
      subject to Rice Theorem limits (cannot know what the code will
      do), so to prevent another AGI, that 1st AGI might have to limit
      a lot of Turing engines (computers) everywhere in fairly drastic
      and effective ways.
   </p>
   <p>
      This implies that the 1st AGI has very many points of diverse
      contact with the world.  Ie, it can sense most nearly everything
      and moreover, it can actively affect, intervene, and conditionalize
      nearly everything everywhere, as has to be assumed if we are
      going to also assume that the 1st AGI <b>can</b> potentially prevent
      another AGI from coming into being.  Ie, if the requirement is
      "prevent another AGI", then the 1st AGI will have to develop
      very strong and significant capability and power so as to
      ensure itself as a singleton.  Ie, it is a very strong instrumental
      incentive for the 1st AGI to seek and acquire power, so that it
      can maintain/sustain itself as a singleton AGI.
   </p>
   <p>
      However, insofar as this singleton AGI has contact with the world
      in diverse locations, in both a sensory (input) and effector
      (output) aspects, not to mention the level of compute necessary
      to support the level of prediction necessary to prevent other
      AGI from coming into being, we can therefore know for sure that
      this 1st AGI is vast.  Ie, it will for sure have lots and lots
      of component sensors, compute processing components, and actuators.
      Moreover, these input/compute/output components will be diverse
      and have spread and impact over much of the world.  Moreover,
      being that everything made of matter and atoms will undergo
      varying states of decay (via simple rust, occasional breakage,
      wear, etc), it can also be assured that there is some support
      and infrastructure for both building and maintaining all of
      these components and parts which together make the "one" AGI.
   </p>
   <p>
      Moreover, insofar as there is a regular build and extending
      of the sensor/compute/actuator capability, due to power seeking
      and also due to the simple need for repair and improvement,
      there will be a kind of ongoing design update associated
      with replacements of all of these parts.  If a new sensor
      design is discovered by the AGI, it will probably want to
      replace older less efficient/accurate designs with the new
      and improved ones.  As such, there is a kind of ongoing
      cycling of parts, with different parts with differential
      capabilities in a kind of "market" in which the designs
      "compete" for viability to exist, to be built, etc.
   </p>
   <p>
      In regards to the parts/components of the overall single
      AGI, this is enough for the substrate-needs convergence
      arguments to actually apply.
   </p>
   <p>
      For the many components of that AGI (sensors, effectors, etc)
      to continue to be useful, they will themselves have to
      integrate an ever larger fraction of the total truth of
      the relevant parts of physics.  Ie, if a given 'part' does
      not work because it does not do what is needed to work,
      then the part will not work, and it will be replaced by
      one that does -- ie, one that does follow the laws of physics,
      using available atoms/materials, etc.  As such, only those
      parts that are actually compatible with both physics <b>and</b>
      the functional needs of the AGI, will continue to be made
      and be used, repaired, etc.  Hence both need/intention and
      the reality of the lawfulness of the physical universe
      get incorporated with increasing convergent effectiveness
      over the evolution of various possible part designs.
      Moreover, this sort of reasoning applies to <b>all</b> of the
      parts in the AGI.
   </p>
   <p>
      The one AGI becomes, along with the rest of the physical world,
      inclusive of ambient truths of the lawfulness of physics,
      the context and environment in which "part design selection"
      and "existence implementation" occurs.  Ie, these are the
      mate selection and the survival selection dynamics respectively
      of a somewhat more abstracted core notion of what the math
      of evolution itself actually implies.  In this sense, it
      is a kind of evolution that allows for and accounts for
      those parts (sensors, processors) continuing to be a part
      of the overall AGI.  How the AGI parts "replicate themselves"
      and "sustain themselves" in the context of the AGI
      and world environment.
   </p>
   <p>
      It is not a "natural" selection -- there is nothing organic
      about it at all.  It is _artificial_ selection of AGI internal
      parts by the AGI itself, in <b>necessary</b> and <b>irreducible</b>
      cooperation with, and integration of, the ambient laws
      of physics and world context -- ie, not just those things
      you can model in math (ie, complex systems involving actual
      uncertainty are not able to be used to predict real physical
      systems in detail, only maybe approximately, so fine details
      of the real world become relevant in practice and integrated
      into the AGI overall design).  As such, insofar as the AGI
      parts are evolving and being extended to and over the world,
      the AGI is increasing itself (reproducing itself, via the
      extending of itself to new places and niches in the world),
      and undergoing a kind of internal evolution, even though
      it is not "in competition with" any other AGI instances.
   </p>
   <p>
      However qualified, evolution is evolution, and therefore
      the substrate needs convergence argument does apply,
      and is functionally specifically required due to the
      very fact of it being/remaining a singleton in the 1st place.
      Therefore, if actually a singleton, and enforced to remain
      that way, the AGI undergoes internal evolution.  If <b>not</b>
      maintaining self exclusivity, then multiple distinct AGI
      agents eventually emerge, and you then end up with external
      evolution.  Either way, over the long term, substrate needs
      convergence <b>does</b> inexorably happen.
   </p>
   <p>
      Moreover, no control mechanism can even so much as
      slow that substrate needs convergence down, insofar as
      <b>any</b> engineering control methodology will itself be made
      of functional parts and components, which themselves are
      (cannot not be) at once sensors, processors, and actuators --
      ie, as having exactly the same nature of the AGI itself,
      and thus subject to the same laws of physics integration also,
      and thus become <b>contributing</b> to that substrate needs
      convergence process, rather than slowing/halting it.
      If the control mechanisms exist and maintain themselves
      and have actual causative impact in the world, then the
      world will influence (cannot not determinately influence)
      the actual _fact_ of those control mechanisms in terms of
      <b>both</b> sustainability and evolution.  QED.
   </p>
   <p>
      ~ ~ ~
   </p>
   <p>
      If you want/need to send us an email,
      with questions, comments, etc,
      on this or any other topic,
      or on related matters,
      use this address:
   </p>
   <p>
        ai@mflb.com
   </p>
   <p>
      (@ Mode Switch com.op_mode_tog_1();) + (@ View Source com.op_notepad_edit_1();)
   </p>
   <p>
      Back to the (@ Area Index https://mflb.com/ai_alignment_1/index.html).
   </p>
   <p>
   LEGA:
   </p>
   <p>
      Copyright (c) of the non-quoted text, 2022,
      by Forrest Landry.
   </p>
   <p>
      This document will not be copied or reproduced
      outside of the mflb.com presentation context,
      by any means, without the expressed permission
      of the author directly in writing.
      No title to and ownership of
      this or these documents
      is hereby transferred.
   </p>
   <p>
      The author assumes no responsibility
      and is not liable for any interpretation
      of this or these documents
      or of any potential effects and consequences
      in the lives of the readers of these documents.
   </p>
</body>
</html>