Of course, this does not happen in a vanilla Torque 3D environment, but heavily and often changing the source can leave you with a network issue that is very hard to find and might pop up only months after the bug had been planted.
So run this php script on your source directory, but set the value of $sourcedir with a trailing slash (/) first, and then wait for it to execute. It will generate an unbalanced.txt file for you in the source root which you can check for a report.
The output will look something like this:
1 2 3 | // 1 Bitcount mismatch between packData [3532] and unpackData [3539] in X:/Xenocell/engine/source/T3D/fx/explosion.cpp.// 2 Bitcount mismatch between pack [72] and unpack [68] in X:/Xenocell/engine/source/T3D/fx/lightning.cpp.// 3 Bitcount mismatch between packUpdate [268] and unpackUpdate [267] in X:/Xenocell/engine/source/T3D/item.cpp. |
The following is an excerpt from my resource post over at GarageGames.com:
And the PHP source code. Disclaimer: it was written quickly out of an urgent necessity, but it works :)Like all static code analyzers, this is not 100% exact, but to my defense, it will almost surely find it if bit counts mismatch in ie. a packUpdate and an unpackUpdate. The downside is that it reports a few false positives. I had to leave these in to not decrease the chance of it finding a real problem.The script goes through all .cpp files in your source directory and checks whether they have pack/unpack, packData/unpackData, packUpdate/unpackUpdate and readPacketData/writePacketData methods, and if they do, it will compare the stream i/o commands used. It tries to find the number of bits used in each operation where possible, or assign some realistic value.The downside if that it does not currently compare the order of these commands, but being able to find an issue where you write one bit more than you read is a pretty good thing in itself if I may say so.False positives are usually cases, where the program structures to write and to read bits into the bitstream are different. (Ie. using a writeFlag(true) and a writeFlag(false) in two branches of a condition which is read by a readFlag()) There are currently about 6 such reports in vanilla Torque 3D 1.2 and 6+1 in AFX for Torque 3D 1.2. So if you get mismatches over these values, be very suspicious! :)If you intend to use this tool to check on your code from time to time, you might want to rewrite the parts where false positives are reported, so that they don't mislead you the next time.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 | <?php//--------------------------------------------------------------------------------------// Static Network BitStream Analyzer for Torque 3D (c) 2012 Bitgap Games by Konrad Kiss//--------------------------------------------------------------------------------------$sourcedir = "X:/Xenocell/engine/source/";error_reporting(E_ALL);function errorhandler($errno, $errstr, $errfile, $errline) { echo($errstr." in ".$errfile." on line ".$errline); exit;// return true;}$old_error_handler = set_error_handler("errorhandler");$cmdCosts = array( // write fn => array(regexp, costParamPos (1..) or 0, defaultCost) "writeInt" => array("stream\-\>writeInt[\W]*\(([^,\n]+)\,([^)\n]+)\)", 2, 32), "readInt" => array("stream\-\>readInt[\W]*\(([^)\n]+)\)", 1, 32), "mathWrite" => array("mathWrite[\W]*\([\W]*\*[\W]*stream[\W]*\,([^)\n]+)\)", 0, 64), "mathRead" => array("mathRead[\W]*\([\W]*\*[\W]*stream[\W]*\,([^)\n]+)\)", 0, 64), "writeCussedU32" => array("stream\-\>writeCussedU32[\W]*\(([^)\n]+)\)", 0, 37), "readCussedU32" => array("stream\-\>readCussedU32[\W]*\(([^)\n]*)\)", 0, 37), "writeSignedInt" => array("stream\-\>writeSignedInt[\W]*\(([^,\n]+)\,([^)\n]+)\)", 2, 32), "readSignedInt" => array("stream\-\>readSignedInt[\W]*\(([^)\n]+)\)", 1, 32), "writeRangedU32" => array("stream\-\>writeRangedU32[\W]*\(([^,\n]+)\,([^,\n]+)\,([^)\n]+)\)", 0, 32), "readRangedU32" => array("stream\-\>readRangedU32[\W]*\(([^,\n]+)\,([^)\n]+)\)", 0, 32), "writeRangedS32" => array("stream\-\>writeRangedS32[\W]*\(([^,\n]+)\,([^,\n]+)\,([^)\n]+)\)", 0, 32), "readRangedS32" => array("stream\-\>readRangedS32[\W]*\(([^,\n]+)\,([^)\n]+)\)", 0, 32), "writeFloat" => array("stream\-\>writeFloat[\W]*\(([^,\n]+)\,([^)\n]+)\)", 2, 32), "readFloat" => array("stream\-\>readFloat[\W]*\(([^)\n]+)\)", 1, 32), "writeSignedFloat" => array("stream\-\>writeSignedFloat[\W]*\(([^,\n]+)\,([^)\n]+)\)", 2, 32), "readSignedFloat" => array("stream\-\>readSignedFloat[\W]*\(([^)\n]+)\)", 1, 32), "writeRangedF32" => array("stream\-\>writeRangedF32[\W]*\(([^,\n]+)\,([^,\n]+)\,([^,\n]+)\,([^)\n]+)\)", 4, 32), "readRangedF32" => array("stream\-\>readRangedF32[\W]*\(([^,\n]+)\,([^,\n]+)\,([^)\n]+)\)", 3, 32), "writeClassId" => array("stream\-\>writeClassId[\W]*\(([^,\n]+)\,([^,\n]+)\,([^)\n]+)\)", 0, 32), "readClassId" => array("stream\-\>readClassId[\W]*\(([^,\n]+)\,([^)\n]+)\)", 0, 32), "writeNormalVector" => array("stream\-\>writeNormalVector[\W]*\(([^,\n]+)\,([^)\n]+)\)", 2, 96), "readNormalVector" => array("stream\-\>readNormalVector[\W]*\(([^,\n]+)\,([^)\n]+)\)", 2, 96), "writeCompressedPoint" => array("stream\-\>writeCompressedPoint[\W]*\(([^,\n]+)\,([^)\n]+)\)", 0, 32), "readCompressedPoint" => array("stream\-\>readCompressedPoint[\W]*\(([^,\n]+)\,([^)\n]+)\)", 0, 32), "writeVector" => array("stream\-\>writeVector[\W]*\(([^,\n]+)\,([^,\n]+)\,([^,\n]+)\,([^)\n]+)\)", 4, 32), "readVector" => array("stream\-\>readVector[\W]*\(([^,\n]+)\,([^,\n]+)\,([^,\n]+)\,([^)\n]+)\)", 4, 32), "writeAffineTransform" => array("stream\-\>writeAffineTransform[\W]*\(([^)\n]+)\)", 0, 96), "readAffineTransform" => array("stream\-\>readAffineTransform[\W]*\(([^)\n]+)\)", 0, 96), "writeQuat" => array("stream\-\>writeQuat[\W]*\(([^,\n]+)\,([^)\n]+)\)", 2, 9), "readQuat" => array("stream\-\>readQuat[\W]*\(([^,\n]+)\,([^)\n]+)\)", 2, 9), "writeQuat2" => array("stream\-\>writeQuat[\W]*\(([^,\n]+)\)", 0, 9), "readQuat2" => array("stream\-\>readQuat[\W]*\(([^,\n]+)\)", 0, 9), "writeBits" => array("stream\-\>writeBits[\W]*\(([^,\n]+)\,([^)\n]+)\)", 1, 1), "readBits" => array("stream\-\>readBits[\W]*\(([^,\n]+)\,([^)\n]+)\)", 1, 1), "writeBits2" => array("stream\-\>writeBits[\W]*\(([^)\n]+)\)", 0, 1), "readBits2" => array("stream\-\>readBits[\W]*\(([^)\n]+)\)", 0, 1),// "writeFlagTrue" => array("stream\-\>writeFlag[\W]*\([ ]*true[ ]*\)", 0, 0), // trick to not have to parse conditions// "writeFlagFalse" => array("stream\-\>writeFlag[\W]*\([ ]*false[ ]*\)", 0, 0), // trick to not have to parse conditions "writeFlag" => array("stream\-\>writeFlag[\W]*\(([^)\n]+)\)", 0, 1), "readFlag" => array("stream\-\>readFlag[\W]*\(([^)\n]*)\)", 0, 1), "readFlag2" => array("_readDirtyFlag[\W]*\([\W]*stream[\W]*\,([^)\n]+)\)", 0, 1), // sfxEmitter exotics "writeString" => array("stream\-\>writeString[\W]*\(([^,\n]+)\,([^)\n]+)\)", 1, 256*8), "writeString2" => array("stream\-\>writeString[\W]*\(([^)\n]+)\)", 0, 256*8), "readString" => array("stream\-\>readString[\W]*\(([^)\n]+)\)", 0, 256*8), "readSTString" => array("stream\-\>readSTString[\W]*\(([^)\n]*)\)", 0, 256*8), "write" => array("stream\-\>write[\W]*\(([^)\n]+)\)", 0, 64), "read" => array("stream\-\>read[\W]*\(([^)\n]+)\)", 0, 64),);$errorcount = 0;$currentFile = "";$output = "";$summary = "";uecho("// ****************************************************************\n");find_files($sourcedir, "/cpp$/", "examine");file_put_contents($sourcedir."unbalanced.txt", "// SUMMARY\n\n\n".$summary."\n\n\n\n// DETAILED OUTPUT".$output);uecho("// ****************************************************************\n");//------------------------------------------------------------------------------function fn_countbits($source, $bitstreamName)//------------------------------------------------------------------------------{ global $cmdCosts, $currentFile; $lines = explode("\n", $source); $count = 0; $cmdlist = ""; for ($l=0;$l<count($lines);$l++) { //uecho("."); $line = $lines[$l]; // strip comments foreach ($cmdCosts as $cmd => $data) { $pattern = "/".str_replace("stream", $bitstreamName, $data[0])."/s"; preg_match($pattern, $line, $params); if (count($params)>0) { if ($data[1]>0) { // there is a parameter that explicitly defines the number of used bits for this operation if (is_numeric($params[$data[1]])) { // the cost is a clear numeric bit count $cost = $params[$data[1]]; } else { // the cost could be a variable or a function perhaps - we'll default to a cost of 1 here $cost = 1; } } else { // no paremeters are used to define bitcount... we can then use a default bitcount for comparison // it could be 0, which means that the command should not be used to compare values // It is used in the often case when writing a flag is broken up into the two branches of a condition while // only one command is reading the other side. $cost = $data[2]; } //uecho($cmd." : ".$cost."\n");//." [".$data[1]." / ".$params[$data[1]]." / ".$data[2]."]\n"); $count += $cost; $cmdlist .= $cmd." - ".$cost." @ line ".$l."\n"; if ($cmd=="writeFlagTrue" || $cmd=="writeFlagFalse") break; } } //uecho(trim($lines[$l]," \t\n\r")."\n"); } //$cmdcount = substr_count($source, $bitstreamName."->"); return array($count, $cmdlist);}//------------------------------------------------------------------------------function compare($filepath, $fn1, $fn2)//------------------------------------------------------------------------------{ global $errorcount; $content = file_get_contents($filepath); // remove comments (basic, but mostly does the job) $content = preg_replace('/(\/\/[^\n]*\n)/sm', "\n", $content); $content = preg_replace('/(\/\*.*?\*\/)/sm', "", $content); // get rid of all return characters $content = str_replace("\r", "", $content); // replace multiple new lines with a single new line $content = preg_replace('/([\n]+)/sm', "\n", $content); // remove irrelevant lines $wasif = true; $wassec = true; $wasfn = true; $wasrw = true; $wassd = false; $lvl = 0; $modified = true; $newcontent = ""; foreach (explode("\n", $content) as $linenum => $line) { //uecho("*"); // replace multiple tabs and spaces with single spaces //$line = preg_replace('/([\t ]+)/sm', " ", $line); //$line = trim($line, " \t"); $line = rtrim($line, " \t"); // remove strings between "" //$line = preg_replace('/(\")[^\"]*(\")/', "", $line); if ($line=="") { continue; } $keep = false; $if0 = (strpos($line, "if (") !== false || strpos($line, "if(") !== false ); $if1 = (strpos($line, "else") !== false ); $if = ($if0 || $if1); $sec =(strpos($line, "{") !== false || strpos($line, "}") !== false ); $fn = (strpos($line, "::".$fn1) !== false || strpos($line, "::".$fn2) !== false ); $rw = (strpos($line, "->write") !== false || strpos($line, "->read") !== false || strpos($line, "mathRead") !== false || strpos($line, "mathWrite") !== false ); $sd = strpos($line, ";") !== false; $sp = strpos($line, "(") !== false; $ep = strpos($line, ")") !== false; $separator = "\n"; if (!$wassd && !$wassec && !$if && !$rw) { $separator = " "; $line = trim($line, " \t"); $modified = true; } $newcontent .= $separator.$line; $wasif = $if; $wassec = $sec; $wasfn = $fn; $wasrw = $rw; $wassd = $sd; } $content = $newcontent; // remove empty {}-s //$content = preg_replace('/(\{[ \t\n\r]*\})/sm', "", $content); $fnone = fn_contents($content, $fn1); $fntwo = fn_contents($content, $fn2); if ($fnone === false && $fntwo == false) return; if (($fnone === false || $fntwo === false)) { $errorcount++; uecho("\n\n\n"); uecho("// ".$errorcount." Missing either a(n) ".$fn1." or a(n) ".$fn2." method in ".$filepath.".\n", true); return; } $one = fn_countbits($fnone[1], $fnone[0]); $two = fn_countbits($fntwo[1], $fntwo[0]); if ($one[0] != $two[0]) { $errorcount++; uecho("\n\n\n"); uecho("// ".$errorcount." Bitcount mismatch between ".$fn1." [".$one[0]."] and ".$fn2." [".$two[0]."] in ".$filepath.".\n", true); uecho("\n// ".str_pad($fn1." (".$fnone[0]." - ".$one[0].") ", 66, "+")."\n"); uecho($fnone[1]); uecho("\n\n".$one[1]); uecho("\n// ".str_pad($fn2." (".$fntwo[0]." - ".$two[0].") ", 66, "+")."\n"); uecho($two[1]."\n\n"); uecho($fntwo[1]); uecho("\n// ".str_pad("", 66, "+")."\n"); }}//------------------------------------------------------------------------------function examine($filepath)//------------------------------------------------------------------------------{ global $currentFile; $currentFile = $filepath; compare($filepath, "pack", "unpack"); compare($filepath, "packData", "unpackData"); compare($filepath, "packUpdate", "unpackUpdate"); compare($filepath, "readPacketData", "writePacketData");}//------------------------------------------------------------------------------function fn_contents($content, $functionName)//------------------------------------------------------------------------------{ $fnofs = strpos($content, "::".$functionName); if ($fnofs === false) return; // find the last enter before the function $lastenterpos = strrpos(substr($content, 0, $fnofs), "\n"); $content = substr($content, $lastenterpos+1); $content = str_replace("\"@", "[at]", $content); // we need the @ and ` characters to replace } and { // the at sign is only found in doc references $end = 0; $lvl = 0; $l = strlen($content); for ($p=0;$p<$l;$p++) { $ch = substr($content, $p, 1); if ($ch=="{") { if ($lvl>0) $content = substr_replace($content, "`", $p, 1); $lvl++; } else if ($ch=="}") { if ($lvl>1) $content = substr_replace($content, "@", $p, 1); if ($lvl>0) { $lvl--; if ($lvl==0) { $end = $p; break; } } } } $content = substr($content, 0, $end+1); $pattern = "/(\w+) \b(\w+)::".$functionName."\(([^{]+)\{(.+?)}/s"; preg_match($pattern, $content, $matches); if (count($matches)>0) { $bitstreamName = trim(substr($matches[3], strpos($matches[3], "BitStream")+9), " *)\n\t\r"); $par = strpos($bitstreamName, ")"); if ($par) $bitstreamName = substr($bitstreamName, 0, $par-1); $com = strpos($bitstreamName, ","); if ($com) $bitstreamName = substr($bitstreamName, 0, $com-1); $fnsource = str_replace("`", "{", str_replace("@", "}", $matches[4])); $fnsource = str_replace("[at]", "@", $fnsource); } else { return false; } $res = array(); $res[0] = $bitstreamName; $res[1] = $fnsource; return $res;}//------------------------------------------------------------------------------function find_files($path, $pattern, $callback)//------------------------------------------------------------------------------{ $path = rtrim(str_replace("\\", "/", $path), '/') . '/*'; foreach (glob ($path) as $fullname) { //uecho("%"); if (basename($fullname)=="_svn" || basename($fullname)==".svn") continue; if (is_dir($fullname)) { //uecho("\n".$fullname.":\n", true); find_files($fullname, $pattern, $callback); } else if (preg_match($pattern, $fullname)) { //uecho(" ".$fullname." (" . round(filesize($fullname)/1024, 2) . "kBytes\n"); call_user_func($callback, $fullname); } }}//------------------------------------------------------------------------------function uecho($str, $important=false)//------------------------------------------------------------------------------{ global $output, $summary; if ($important==true) { $summary .= $str; } $output .= $str; echo($str);}?> |
Try changing the bitcount of a single writeInt for example, and see if it catches it!
No comments:
Post a Comment