Re: size of news.dat & yarnpack

From: ---{Q[4]Qo. (gro.oi@mt)
Date: Sat, 04 Jan 1997 21:46:15 -0500

Sun, 05 Jan 1997 00:16:02 +0100, rgriech@ibm.net (Hardy Griech) wrote:
>Interesting features. Does it mean that history.pag contains only a
>hashed list of the articles?

It looks like it contains the MsgID, offset in the News.dat (first four
bytes displayed by my wimpy rexx script), as well as the import date (which
i presume is the second four bytes, but i haven't been able to figure out
the format (i've tried a few julian calculations and DOS bit packed
format... haven't tried to get it real hard, but have tried a few things to
no avail). Then there are four more bytes which seldom vary from 0 ... i
believe they are the "expires" date, for messages with such a header.)

We have to draw Chin into this conversation to shed some light on these
trivial technicalities -- or perhaps he enjoys sitting back watching silly
ignorant hackers like myself fumbling around in the dark. (-: Re-inventing
the wheel.

Such an adventure. Maybe a little mystery has it's uses too. heh.

You know, quite possibly he just has some library packaged routine he
picked up somewhere to do all this hashing stuff... who knows.

Hmmm, my news.dat did actually corrupt today... interesting. heh. Not
good... difficult to discern why, though...

I got the old...

Assertion failed: blk.prev + free.size + sizeof(BlockHeader) == delOffset,
file ..\SPOOLFIL.CPP, line 235
ABNORMAL TERMINATION

Wish i knew what that means exactly. (-: It occurs after expire seems to
have done everything... after all the newsgroups have been listed... and
things are evidentally being updated for a while... then boom...

I run expire -o again right after and no more message expire -- so the
first expiry run must have gotten them all (maybe). But something has to
have been not updated....

I run NNIGN and it runs into no problems, so it seems the integrety of the
news.dat, the articles at least, are intact.

So i backed everything up and ran my YARNPACK just to see what happens!! My
thoery is in that absence of facts, flounder around and see what happens!
(-:

Well, YARNPACK ran fine and dandy, no problem. I ran NNIGN after that, and
found another divide by zero error in it (is there no end of them?!), which
i fixed. Then NNIGN ran ok... the newsdat looks fine and healthy. YARNPACK
had squeezed out just over 4 megs of slack, reducing the news.dat to about
9.5 megs. So i ran expire -o once again.. to see what would happen.

So as i was saying i ran expire -o again... 12 more messages had aged out
by this time... expire reports that 335820 bytes have been deleted... and
finished normally. NO ERROR MESSAGE anymore. It appears that YARNPACK has
actually FIXED the news.dat. VERY INTERESTING. (-:

I can only hypothosize (and Chin can remain silent if i'm right to confirm
it. Heheh) that the the free space links in the news.dat got messed up
somehow (each "free" space block has an extra header to build a double
linked list) -- yarnpack eliminates all free space and so there are no free
space links left.. the problem is solved. JUST A THEORY, based on pure
supposition!

(perhaps, as it's last step, that's when expire goes back and starts
consolidating all the free blocks... joining them together... perhaps
that's where it ran into the problem.... doing that, trying to do that...
lots of guesses on my part)

I ran Yarn, scooted around my news... looking at old messages, and
newsgroups likely to have the newest messages, and some inbetween... random
checks... everything seems to be in working order...

Interesting eh... of course i'm paranoid that YARNPACK caused the problem
somehow in teh first place (no idea how -- and people who have never used
yarnpack (and hardly anyone has!) get this problem as we've seen posted
many times in the past -- so i shouldn't jumpt to conclusions and say
Yarnpack caused it yet -- who knows)... but now it seems that whatever
caused it, Yarnpack has solved it, without having to rebuild.

Of course it coudl be all an illusion and problems compounded and made
worse will surface later. Who knows, who knows! (-:

What fun.

The experiments continue... come, Igor... back to the laboratory!

-- 
 _____________________________________________________________________
  Tim Muddleten =-=-=- with love and squalor -=-=-= as544 torfree.net
  -=- drop everything & read The Screwtape Letters by C. S. Lewis -=-
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Yarn/2 Bells & Whistles Page: http://www.io.org/~tm/bells2.html
        * January 4th * International Day of the Reboot Button