How to find corrupt documents with "Field is too large (32k) or View’s column or selection formula are too large" error
Julian Buss, August 13th, 2008 15:08:23
Tags:  LotusNotes 
We just had a case where a 20GB database had some documents with text fields containing more than 32k. Due to these documents the database could not be compacted and converted to R8 format, which was neccessary to activate the "allow more fields" property.  I have no idea what caused this problem, how these corrupt documents were created. Our first priority was to fix this database.

So the big question is: how do you find these corrupt documents in a 20GB database?
  • fixup did not help.
  • compact did not help.
  • compact -c caused the whole server to crash.
  • compact -c while Domino was down crashed, too.
  • ScanEZ was no help: it could scan the database for documents and reported the "Field is too large..." error message, but did not reported on which NoteID that happened.
  • An agent running through all documents and printing out name and size of all items for each document ran without error.
  • Creating a local replica just failed at some point during the replication, with the "Field is too large..:" message, but without reporting the NoteID in the log.

Finally the update task was our rescue. We removed the full text index and let the update task create a new one with "load updall -X path/database.nsf". And while update worked on the database, it reported serveral "error full text indexing..." messages to the log, and finally these errors contained the NoteIDs of the corrupt documents.

After we got the NoteIDs removing the documents was a piece of cake. After that we could successfully create a local replica and we will exchange the database on the server with that replica, then the problem should be finally solved.

Comments (8) | Permanent Link

Kommentare:

1) How to find corrupt documents with Field is too large (32k) or View’s column or selection formula are too large error
Jack Dausman 14.08.2008 21:46:09

Hm.{ Link }

I wonder if the log_update=2 would have also provided the information ?

2) How to find corrupt documents with Field is too large (32k) or View’s column or selection formula are too large error
Jez McKinley 22.03.2009 23:39:43

I take my hat off to you sir! Many thanks.

I was having a big problem with replication between 2 servers where the replication would transfer 53M every replication rather than the <300K normally seen. It would also take nearly an hour rather than the 10secs it usually took.

I had seen this before being down to a corrupt doc, but in this case all I was getting in the logs was 'partially replicated due to a previously reported error' which helpfully had never been reported. fixup and compact did not find any problems. The db was not full text indexed anyway, but I thought I'd try it, and bingo, found the doc, deleted it and we are back to normal with replication.

Thanks again,

Jez.

3) How to find corrupt documents with Field is too large (32k) or View’s column or selection formula are too large error
Maria 08.05.2009 17:29:49

Hi Julian,

We're facing the same problem and we've already have tried all suggested.

Once we have identificated the corrupted documents we cannot delete them because 32k error is raise.

Do you have any ideas?.

Thanks in Advance.

4) How to find corrupt documents with Field is too large (32k) or View’s column or selection formula are too large error
Julian Buss 11.05.2009 08:37:27

use ytria scanEZ to remove the document once you have the noteID / UNID

5) Fixup -V
Michael N 01.12.2011 19:05:22

Fixup -V helps (Domino 8.5.2FP3), all other tips failed, even ScanEZ couldn't delete the document.

12/01/2011 06:32:41 PM Remote console command issued by Michael/415/INT/EEC: Load Fixup "database.nsf" -V

12/01/2011 06:33:17 PM Queuing compact command for database.nsf by Michael/415/INT/EEC

12/01/2011 06:33:18 PM Document NT000216C6 in database D:\Lotus\Domino\Data\database.nsf is damaged: Field is too large (32K) or View's column & selection formulas are too large

12/01/2011 06:33:18 PM Document (UNID OF3E4B8880:85F8FDB7-ONC1257952:00348576) in database D:\Lotus\Domino\Data\database.nsf has been deleted

12/01/2011 06:33:29 PM Completed consistency check on database.nsf

12/01/2011 06:33:29 PM Database Fixup: Shutdown

6) How to find corrupt documents with Field is too large (32k) or View’s column or selection formula are too large error
Julian Buss 02.12.2011 09:22:58

thanks for the tip, that didn't work in my previous cases... I will try it on 8.5.3 when such problem arises again.

7) How to find corrupt documents with Field is too large (32k) or View’s column or selection formula are too large error
Peter Groh 05.04.2012 08:19:02

Interesting thread ...

But .. why do you want to delete these documents ?

Are they worthless ?

I would have suggested an agent which would have set the summary flag of the item to false ... so - this item would be able to store text upto 64k.

And next thing .. if the item has to be shown in views ... cut value at 30k and insert rest of values in another field which probably same name ...

OK .. for sure anything to do for a workaround ...

but only worth if the docs have to be kept !

Best regards

8) How to find corrupt documents with Field is too large (32k) or View’s column or selection formula are too large error
Julian Buss 05.04.2012 08:30:05

Peter, you cannot open these kind of documents anymore. The only thing you can do with them is to delete them.

 Erstellen Sie einen Kommentar
Thema:
   
Name:
E-mail:
Web:
 
Kommentar:  (Kein HTML - URLs bitte mit führendem http:// angeben)
 
Angaben merken?    
Über diese Seite
Das YouAtNotes Blog ist ein Webtagebuch der Mitarbeiter von YouAtNotes. Hier möchten wir ihnen direkt aus dem Alltag eines Softwarehauses berichten.
Wir freuen uns über Ihre Kommentare und wünschen Ihnen anregende und unterhaltsame Lektüre.

Impressum

Blog of Julian Buss

Suche
RSS-Feeds
rssfeed.pngContent Feed
rssfeed.pngComment Feed
Aktuelle Einträge
Aktuelle Kommentare
Links zu YouAtNotes
Links zu YouAtNotes (in englisch)
Archiv