[TASK] Update readme of EXT:lowlevel
[Packages/TYPO3.CMS.git] / typo3 / sysext / lowlevel / README.rst
1 Introduction
2 ============
3
4 For various reasons your TYPO3 installation may over time accumulate data with integrity problems or data you wish
5 to delete completely. For instance, why keep old versions of published content? Keep that in your backup - don't load
6 your running website with that overhead!
7
8 Or what about deleted records? Why not flush them - they also fill up your database and filesystem and most likely you
9 can rely on your backups in case of an emergency recovery?
10
11 Also, relations between records and files inside TYPO3 may be lost over time for various reasons.
12
13 If your website runs as it should such "integrity problems" are mostly easy to automatically repair by simply removing
14 the references pointing to a missing record or file.
15
16 However, it might also be "soft references" from eg. typolinks (`<link 123>...</link>`) or a file references in a
17 TypoScript template (`something.file = fileadmin/template/miss_me.jpg`) which are missing. Those cannot be automatically
18 repaired but the cleanup script incorporates warnings that will tell you about these problems if they exist and you
19 can manually fix them.
20
21 These scripts provides solutions to these problems by offering an array of tools that can analyze your TYPO3 installation
22 for various problems and in some cases offer fixes for them.
23
24
25
26 Preparations
27 ============
28
29 THERE IS ABSOLUTELY NO WARRANTY associated with this script! It is completely on your OWN RISK that you run it.
30 It may cause accidental data loss due to software bugs or circumstances that it does not know about yet - or data
31 loss might happen due to misuse!
32
33 ALWAYS make a complete backup of your website! That means:
34
35 * Dump the complete database to an SQL file. This can usually be done from the command line like this::
36
37 mysqldump [database name] -u [database user] -p --add-drop-table > ./mywebsite.sql
38
39 * Save all files in the webroot of your site. I usually do this from the command line like this::
40
41 tar czf ./mywebsite.tgz [webroot directory of your site]
42
43 It could be a good idea to run a `myisamchk` on your database just to make sure MySQL has everything pulled together right.
44
45 Something like this will do::
46
47 myisamchk [path_to_mysql_databases]/[database_name]/*.MYI -s -r
48
49
50
51 Running the script
52 ==================
53
54 The "[base command]" is::
55
56 [typo3_site_directory]/typo3/sysext/core/bin/typo3
57
58 Try this first. If it all works out you should see a help-screen. Otherwise there will be instructions about what to do.
59
60 You can use the script entirely by following the help screens. However, through this document you get some idea about
61 the best order of events since they may affect each other.
62
63 For each of the tools in the test you can see a help screen by running::
64
65 [base command] --help [toolkey]
66
67 Example with the tool "orphan_records"::
68
69 [typo3_site_directory]/typo3/sysext/core/bin/typo3 --help cleanup:orphanrecords
70
71
72
73 Suggested order of clean up
74 ---------------------------
75
76 The suggested order below assumes that you are interested in running all these tests. Maybe you are not! So you should
77 check the description of each one and if there is any of the tests you wish not to run, just leave it out.
78
79 It kind of gets simpler that way since the complexity mostly is when you wish to run all tests successively in which
80 case there is an optimal order that ensures you don't have to run the tests all over again.
81
82 - `[base command] cleanup:orphanrecords`
83
84 - As a beginning, get all orphaned records out of the system since you probably want to. Since orphan records may
85 keep some missing relations from being detected it's a good idea to get them out immediately.
86
87 - `[base command] cleanup:versions`
88
89 - Flush all published versions now if you like. Published versions may also keep references to records which could
90 affect other tests, hence do it now if you want to.
91
92 - `[base command] cleanup:multiplereferencedfiles`
93
94 - Fix any files referenced twice or more before you delete records (which could potentially delete a file that is
95 referenced by another file).
96
97 - `[base command] cleanup:deletedrecords`
98
99 - Flush deleted records. As a rule of thumb, tools that create deleted records should be run before this one so
100 the deleted records they create are also flushed (if you like to of course)
101
102 - `[base command] cleanup:missingrelations`
103
104 - Remove missing relations at this point.
105 - If you get an error like this; "\TYPO3\CMS\Core\Database\ReferenceIndex::setReferenceValue(): ERROR: No reference
106 record with hash="132ddb399c0b15593f0d95a58159439f" was found!" just run the test again until no errors occur.
107 The reason is that another fixed reference in the same record and field changed the reference index hash. Running
108 the test again will find the new hash string which will then work for you.
109
110 - `[base command] cleanup:flexforms`
111
112 - After the "deleted" tool since we cannot clean-up deleted records and to make sure nothing unimportant
113 is cleaned up.
114
115 - `[base command] cleanup:rteimages`
116
117 - Will be affected by flushed deleted records, versions and orphans so must be run after any of those tests.
118
119
120
121 Executed anytime
122 ----------------
123
124 These can be executed anytime, however you should wait till all deleted records and versions are flushed so you don't
125 waste system resources on fixing deleted records.
126
127 ::
128
129 [base command] cleanup:missingfiles
130 [base command] cleanup:lostfiles
131
132
133 Nightly reports of problems in the system
134 -----------------------------------------
135
136 If you wish to scan your TYPO3 installations for problems with a cronjob or so, a shell script that outputs a
137 report could look like this::
138
139 #!/bin/sh
140 [typo3_site_directory]/typo3/sysext/core/bin/typo3 cleanup:orphanrecords -vv
141 [typo3_site_directory]/typo3/sysext/core/bin/typo3 cleanup:versions -vv
142 [typo3_site_directory]/typo3/sysext/core/bin/typo3 cleanup:multiplereferencedfiles --update-refindex -vv
143 [typo3_site_directory]/typo3/sysext/core/bin/typo3 cleanup:deletedrecords -v
144 [typo3_site_directory]/typo3/sysext/core/bin/typo3 cleanup:missingrelations --update-refindex -vv
145 [typo3_site_directory]/typo3/sysext/core/bin/typo3 cleanup:flexforms -vv
146 [typo3_site_directory]/typo3/sysext/core/bin/typo3 cleanup:rteimages --update-refindex -vv
147 [typo3_site_directory]/typo3/sysext/core/bin/typo3 cleanup:missingfiles --update-refindex -vv
148 [typo3_site_directory]/typo3/sysext/core/bin/typo3 cleanup:lostfiles --update-refindex -vv
149
150
151 You may wish to change the verbosity level from `-vv` to `-v` as in the case above, depending on how important
152 you consider the warnings.
153
154 You might also wish to disable tests like "deleted" which would report deleted records - something that might not
155 warrant a warning, frankly speaking...
156
157 Example script for checking your installation
158 ---------------------------------------------
159
160 ::
161
162 #!/bin/sh
163 ./typo3/sysext/core/bin/typo3 cleanup:orphanrecords -vv
164 ./typo3/sysext/core/bin/typo3 cleanup:versions -vv
165 ./typo3/sysext/core/bin/typo3 cleanup:multiplereferencedfiles -vv
166 ./typo3/sysext/core/bin/typo3 cleanup:deletedrecords -v
167 ./typo3/sysext/core/bin/typo3 cleanup:missingrelations -vv
168 ./typo3/sysext/core/bin/typo3 cleanup:flexforms -vv
169 ./typo3/sysext/core/bin/typo3 cleanup:rteimages -vv
170 ./typo3/sysext/core/bin/typo3 cleanup:missingfiles -vv
171 ./typo3/sysext/core/bin/typo3 cleanup:lostfiles -vv
172
173
174 Example script for cleaning your installation
175 ---------------------------------------------
176
177 ::
178
179 #!/bin/sh
180 ./typo3/sysext/core/bin/typo3 cleanup:orphanrecords -vv
181 ./typo3/sysext/core/bin/typo3 cleanup:versions -vv
182 ./typo3/sysext/core/bin/typo3 cleanup:multiplereferencedfiles -vv --update-refindex
183 ./typo3/sysext/core/bin/typo3 cleanup:deletedrecords -v
184 ./typo3/sysext/core/bin/typo3 cleanup:missingrelations -vv --update-refindex
185 ./typo3/sysext/core/bin/typo3 cleanup:flexforms -vv
186 ./typo3/sysext/core/bin/typo3 cleanup:rteimages -vv
187 ./typo3/sysext/core/bin/typo3 cleanup:missingfiles --update-refindex
188 ./typo3/sysext/core/bin/typo3 cleanup:lostfiles -vv --update-refindex
189