Panic on Raspberry Pi

One of my Pi’s went offline a few days ago - when I got access to it yesterday, rebooted it, waited for the web gui to respond - it never did. Went into the .config folder and found 6634 panic files…

here is one of them

    [PHDB3] 12:07:18 INFO: syncthing v0.11.10 (go1.4.2 linux-arm default) unknown-user@syncthing-builder 2015-06-21 09:45:54 UTC
    [PHDB3] 12:07:18 INFO: My ID: ********************************************
    [PHDB3] 12:07:18 INFO: Database block cache capacity 8192 KiB
    Panic at 2015-06-27T12:07:18+01:00
    panic: leveldb: iKey "\x01", len=1: invalid length

    goroutine 1 [running]:, 0x1, 0x23)
	/go/src/ +0x1d4, 0x1, 0x23, 0x0, 0x0, 0x0)
	/go/src/ +0x44*iComparer).Compare(0x10aaa458, 0x10a44ecc, 0x1, 0x23, 0x10af4012, 0x9, 0x9, 0xffffffff)
	/go/src/ +0x38·001(0x5, 0x849900)
	/go/src/ +0x350
    sort.Search(0x6, 0x10adab9c, 0x0)
	/usr/local/go/src/sort/search.go:66 +0x60*block).seek(0x10ee6330, 0x76fb5840, 0x10aaa458, 0x0, 0x6, 0x10af4012, 0x9, 0x9, 0x3f0a30, 0x0, ...)
	/go/src/ +0x9c*blockIter).Seek(0x10e76c80, 0x10af4012, 0x9, 0x9, 0x0)
	/go/src/ +0xe8*Reader).newBlockIter(0x113eb290, 0x10ee6330, 0x76fb6cf0, 0x10a0bbc8, 0x10b3ec00, 0x1, 0x0)
	/go/src/ +0x1d0*Reader).NewIterator(0x113eb290, 0x10b3ec00, 0x0, 0x0, 0x0)
	/go/src/ +0x2e4*tOps).newIterator(0x10a39690, 0x10b49890, 0x10b3ec00, 0x0, 0x0, 0x0)
	/go/src/ +0x160*version).getIterators(0x10b4b380, 0x10b3ec00, 0x0, 0x113e9400, 0x80, 0x80)
	/go/src/ +0xcc*DB).newRawIterator(0x10a3e540, 0x10b3ec00, 0x0, 0x0, 0x0)
	/go/src/ +0x78*DB).newIterator(0x10a3e540, 0xcb786dd, 0x0, 0x10b3eba0, 0x0, 0x10af4011)
	/go/src/ +0x1a0*Snapshot).NewIterator(0x10a2f2f0, 0x10b3eba0, 0x0, 0x0, 0x0)
	/go/src/ +0x2e0, 0x0, 0x0, 0x0)
	/go/src/ +0x1f8, 0x0, 0x0, 0x0)
	/go/src/ +0x34
	/go/src/ +0x17e0
	/go/src/ +0x1d28

goroutine 5 [syscall]:
	/usr/local/go/src/os/signal/signal_unix.go:21 +0x1c
created by os/signal.init·1
	/usr/local/go/src/os/signal/signal_unix.go:27 +0x40

goroutine 7 [chan receive]:
	/go/src/ +0x120
created by main.init·2
	/go/src/ +0x34

goroutine 8 [select]:*Supervisor).Serve(0x10adc180)
	/go/src/ +0xd60
created by*Supervisor).ServeBackground
	/go/src/ +0x3c

goroutine 10 [select]:*Subscription).Poll(0x10a38e10, 0xf8475800, 0xd, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/ +0x2b8*BufferedSubscription).pollingLoop(0x10a2e990)
	/go/src/ +0x40
created by
	/go/src/ +0x2a8

goroutine 11 [select]:*BufferPool).drain(0x10a6e420)
	/go/src/ +0x1d4
created by
	/go/src/ +0x23c

goroutine 12 [select]:*DB).compactionError(0x10a3e540)
	/go/src/ +0x474
created by
	/go/src/ +0x794

goroutine 13 [select]:*DB).mpoolDrain(0x10a3e540)
	/go/src/ +0x124
created by
	/go/src/ +0x7bc

goroutine 14 [runnable]:
syscall.Syscall6(0xb4, 0xd3, 0x10f64000, 0x10b, 0x0, 0x292a03, 0x0, 0x10b, 0x10f64000, 0x0)
	/usr/local/go/src/syscall/asm_linux_arm.s:48 +0x8
syscall.Pread(0xd3, 0x10f64000, 0x10b, 0x2ef, 0x292a03, 0x0, 0x10b12340, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_linux_arm.go:1835 +0xa8
os.(*File).pread(0x10bbf238, 0x10f64000, 0x10b, 0x2ef, 0x292a03, 0x0, 0x60948, 0x0, 0x0)
	/usr/local/go/src/os/file_unix.go:201 +0x6c
os.(*File).ReadAt(0x10bbf238, 0x10f64000, 0x10b, 0x2ef, 0x292a03, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/os/file.go:117 +0xb4*Reader).readRawBlock(0x10f5c090, 0x292a03, 0x0, 0x106, 0x0, 0x292a01, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/ +0xe8*Reader).readBlock(0x10f5c090, 0x292a03, 0x0, 0x106, 0x0, 0x1, 0x1, 0x0, 0x0)
	/go/src/ +0x5c*Reader).readBlockCached(0x10f5c090, 0x292a03, 0x0, 0x106, 0x0, 0x3f0001, 0x10b9b720, 0x0, 0x0, 0x0, ...)
	/go/src/ +0x22c*Reader).getIndexBlock(0x10f5c090, 0x0, 0xb70ee08b, 0x0, 0x0, 0x0, 0x0)
	/go/src/ +0x7c*Reader).NewIterator(0x10f5c090, 0x0, 0x10b82090, 0x0, 0x0)
	/go/src/ +0x1b4*tOps).newIterator(0x10a39690, 0x10b1cc90, 0x0, 0x10b82090, 0x0, 0x0)
	/go/src/ +0x160*compaction).newIterator(0x10bc2000, 0x0, 0x0)
	/go/src/ +0x268*tableCompactionBuilder).run(0x10aa4120, 0x10b82078, 0x0, 0x0)
	/go/src/ +0x400*DB).compactionTransact(0x10a3e540, 0x548d50, 0xb, 0x76fb6d88, 0x10aa4120)
	/go/src/ +0x21c*DB).tableCompaction(0x10a3e540, 0x10bc2000, 0x0)
	/go/src/ +0xfbc*DB).tableAutoCompaction(0x10a3e540)
	/go/src/ +0x4c*DB).tCompaction(0x10a3e540)
	/go/src/ +0x2d8
created by
	/go/src/ +0x9a4

goroutine 15 [select]:*DB).mCompaction(0x10a3e540)
	/go/src/ +0x248
created by
	/go/src/ +0x9cc

goroutine 16 [select]:*DB).jWriter(0x10a3e540)
	/go/src/ +0x17c
created by
	/go/src/ +0x9f4

Any advice? I have a feeling I will have to delete the index files to make it work again?

Yeah try that, but this is the first time I’ve seen this, so perhaps worth an issue on github. - bug reported here

Same pi, 3004 panic files, it dropped out of sync 5 days ago - I had deleted the index to try and let it rebuild. not sure if it ever managed to

Not all panic files are the same, the first one is very large, best way of uploading it?


This was the first panic … I can give you a sample of some of the others

Most recent:

The other ones are just corruption. You can search the forums/github tracker on how to resolve that.

What I found was just to delete the indexes and start again? But this is how i got back into the situation? Is the first one also just corruption?

The first one is strange, I have never seen it before, hence I cannot say why it happens. If you are constantly getting panics about corrupt indexes, it might be to hardware/software issues outside of syncthing, as explained in the threads.