Naturally algorithm dependant to a point. But also obviously it's still
gonna take longer when you have more data -- no matter how good your
algorithm is to find what is needed. (-: I don't know, of course, how
specifically optimised Yarn is for importing. I do note that the
history.pag is maintained in a somewhat sorted condition. I think it's
sorted on import date though... and also the blocks have the ID's sorted
and grouped together. Presumably the history.dir (which i have no real idea
of the function it serves) may index the history.pag blocks in some sort of
order... probably fairly well optimised whatever it's doing. But still...
more data, more time, of course -- no matter how optimimally indexed. (not
to mention other factors like the tendancy for larger news.dat's, from what
i've seen, to be more fragmented....)
If you want to take a look at a bit of the contents of the history.pag
here's a rexx script that i was playing with then i first started trying to
figure it out.... it will dump the contents, somewhat crudely to stdout...
you need to supply the filename on the command line if you try it... so...
dhash %yarn\history.pag
--
.._,.,._.,.,.,_,._,,,.,_,.,_,..,..,_..,.,.,_,,.,.,_._.,_.,.._,.,_..,.
Tim Middletun =-=-=- with love and squalor -=-=-= as544 torfree.net
-=-=-=- do read Don Quixote by Cervantes if at all possible -=-=-=-
~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~"~
Yarn/2 Bells & Whistles Page: http://www.io.org/~tm/bells2.html
* January 4th * International Day of Spreads
begin 664 dhash.cmd
M+RH@9'5M<"!D871A(&9R;VT@>6%R;B!H87-H(&9I;&4@*"HN<&%G*0T*#0IF
M:6QE;F%M92!O;B!C;VUM86YD(&QI;F4L(')E9&ER96-T(&]U='!U=#H@9&AA
M<V@@:&ES=&]R>2YP86<@/G1E<W0-"@T*<V\@9F%R(&MN;W=N.@T*#0HM(#(@
M:R!B;&]C:W,@*#!X,#@P,"`](#(P-#@I#0HM(&9I<G-T('1W;R!B>71E<R!O
M9B!B;&]C:R!T96QL(&AO=R!M=6-H(&9O;&QO=VEN9R!D871A(&ES('5S960@
M:6X@=&AA="!B;&]C:PT*#0HM('1H:7)D(&)Y=&4@:7,@=&AE(&QE;F=T:"!O
M9B!T:&4@9FER<W0@365S<V%G92U)1`T*+2!T:&5N('1H92!M97-S86=E+4E$
M#0HM(&9I<G-T(&)Y=&4@869T97(@365S<V%G92U)1"!T96QL<R!L96YG=&@@
M;V8@9&%T82!F;VQL;W=I;F<-"BT@9&%T82!F;VQL;W<@;65S<V%G92U)1"`H
M,3(@8GET97,@:70@<V5E;7,@86QW87ES*0T*("`M(&9I<G-T(&9O=7(@8GET
M97,@8FEG(&5N9&EA;B!N97=S+F1A="!O9F9S970@=&\@=&AE(%53140@=F%L
M=64@;V8@:&5A9&5R+@T*("`M('-E8V]N9"!F;W(@8GET97,@;6%Y8F4@9&%T
M92!I;7!O<G1E9#\_("AN;W0@<W5R90T*("`@("TM(&QO;VMS(&)I9R!E;F1I
M86XL(&%N9"!S;W)T960@8GD@=&AA="!F:65L9"D-"B`@+2!L87-T(&9O=7(@
M86QW87ES('-E96T@>F5R;PT**B\-"@T*0FQO8VMS:7IE(#T@,C`T.#L-"@T*
M<&%R<V4@87)G(&9N.PT*#0I3:6=N86P@;VX@;F]V86QU93L-"@T*:68@9FX]
M/2<G('1H96X@9&\-"B`@("!S87D@)VYO(&9I;&5N86UE)SL-"B`@("!E>&ET
M.PT*96YD.PT*#0IF<R`]('-T<F5A;2AF;BPG8R<L)W%U97)Y('-I>F4G*3L-
M"FEF(&9S/&)L;V-K<VEZ92!T:&5N(&1O#0H@("`@<V%Y("=T:&%T(&9I;&4@
M97AI<W1S(&%N9"!I<R!L87)G97(@=&AA;B`G?'QB;&]C:W-I>F5\?"<@8GET
M97,_)SL-"B`@("!E>&ET.PT*96YD#0H-"G1O=&%L8FQO8VMS(#T@*&9S("\@
M8FQO8VMS:7IE*0T*8V%L;"!C:&%R;W5T("=35$1%4E(G+"!T;W1A;&)L;V-K
M<WQ\)R!B;&]C:W,G#0H-"G(@/2!S=')E86TH9FXL)V,G+"=O<&5N(')E860G
M*3L-"G`@/2`P.PT*8FQO8VL@/2`P.PT*#0ID;R!W:&EL92`H8FQO8VL\=&]T
M86QB;&]C:W,I("8@*&-H87)S*&9N*3XQ*0T*("`@('(@/2!S=')E86TH9FXL
M)V,G+"=S965K(#TG?'PH8FQO8VL@*B!B;&]C:W-I>F4I*S$I#0H@("`@8FQO
M8VMU<V5D(#T@8VAA<FEN*&9N+"PR*3L-"B`@("!B;&]C:W5S960@/2!C,F0H
M;&5F="AB;&]C:W5S960L,2DI("L@*&,R9"AR:6=H="AB;&]C:W5S960L,2DI
M("H@,C4V*3L-"B`@("!P(#T@,CL-"B`@("!S87D@8V5N=&5R*"=B;&]C:R`G
M?'QB;&]C:RLQ?'PG("@G?'QB;&]C:W5S961\?"<@8GET97,I)RPU,"PG+2<I
M.PT*("`@(&1O('=H:6QE('`\8FQO8VMU<V5D#0H@("`@("`@(&ED<VEZ92`]
M(&,R9"AC:&%R:6XH9FXL+#$I*3L-"B`@("`@("`@<"`]('`@*R!I9'-I>F4K
M,3L-"B`@("`@("`@:60@/2!C:&%R:6XH9FXL+&ED<VEZ92D[#0H@("`@("`@
M(&1A=&%S:7IE(#T@8S)D*&-H87)I;BAF;BPL,2DI.PT*("`@("`@("!P(#T@
M<"`K(&1A=&%S:7IE*S$[#0H@("`@("`@(&1A=&$@/2!C:&%R:6XH9FXL+&1A
M=&%S:7IE*3L-"B`@("`@("`@<V%Y(&ED#0H@("`@("`@(&AE>'-T<B`]("<G
M#0H@("`@("`@(&1O('(@/2`Q('1O(&1A=&%S:7IE#0H@("`@("`@("`@("!(
M97A3='(@/2!(97A3='(@?'P@<FEG:'0H8S)X*'-U8G-T<BAD871A+'(L,2DI
M+#(L)S`G*7Q\)R`G.PT*("`@("`@("!E;F0[#0H@("`@("`@('-A>2!H97AS
M='([#0H@("`@96YD.PT*("`@(&-A;&P@8VAA<F]U="`G4U1$15)2)RP@)RXG
M.PT*("`@(&)L;V-K(#T@8FQO8VL@*R`Q.PT*96YD.PT*("`@(`T*("`@("`@
J("`@("`@#0H@("`@("`@(`T*("`@("`@("`-"B`@("`-"B`@("`-"@T*
`
end