Package edu.berkeley.nlp.lm.values
Class UncompressedProbBackoffValueContainer
java.lang.Object
edu.berkeley.nlp.lm.values.UncompressedProbBackoffValueContainer
- All Implemented Interfaces:
CompressibleValueContainer<ProbBackoffPair>,ProbBackoffValueContainer,ValueContainer<ProbBackoffPair>,Serializable
public final class UncompressedProbBackoffValueContainer
extends Object
implements ProbBackoffValueContainer
- See Also:
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected final intprotected final long[]protected final booleanprotected final int[]protected booleanprotected final VariableLengthBitCompressorprotected final intprotected final CustomWidthArray[]protected int -
Constructor Summary
ConstructorsConstructorDescriptionUncompressedProbBackoffValueContainer(int valueRadix, boolean storePrefixIndexes, long[] numNgramsForEachOrder, long[] probsAndBackoffsForRank, LongToIntHashMap countIndexer, int wordWidth) UncompressedProbBackoffValueContainer(LongToIntHashMap countCounter, int valueRadix, boolean storePrefixes, long[] numNgramsForEachOrder) -
Method Summary
Modifier and TypeMethodDescriptionbooleanadd(int[] ngram, int startPos, int endPos, int ngramOrder, long offset, long prefixOffset, int word, ProbBackoffPair val_, long suffixOffset, boolean ngramIsNew) Adds a new value at the specified offset.voidclearStorageAfterCompression(int ngramOrder) voidclearStorageForOrder(int ngramOrder) createFreshValues(long[] numNgramsForEachOrder_) Creates a fresh value container for copying purposes.voiddecompress(BitStream bits, int ngramOrder, boolean justConsume, ProbBackoffPair outputVal) Reads and decompresses from the bit stream bits.final floatgetBackoff(int ngramOrder, long index) final floatgetBackoff(CustomWidthArray valueRanksForNgramOrder, long index) getCompressed(long offset, int ngramOrder) Compresses the value at the given offset into a list of bits.protected longgetCountRank(long val) protected ProbBackoffPairvoidgetFromOffset(long index, int ngramOrder, ProbBackoffPair outputVal) Gets the value living at a particular offset.protected voidgetFromRank(long rank, ProbBackoffPair outputVal) final longgetInternalVal(int ngramOrder, long index) final floatgetProb(int ngramOrder, long index) final floatgetProb(CustomWidthArray valueRanksForOrder, long index) protected longgetRank(int ngramOrder, long offset) Creates a fresh value of object (useful for passing as an output parameter)longgetSuffixOffset(long index, int ngramOrder) protected intgetSuffixOffsetFromInternalVal(long internalVal) intnumValueBits(int ngramOrder) voidDestructively sets internal storage from another object.voidsetMap(NgramMap<ProbBackoffPair> map) Initializes a value container with the map that contains itvoidsetSizeAtLeast(long size, int ngramOrder) Sets internal storage for size for a particular n-gram orderbooleanvoidswap(long a, long b, int ngramOrder) Swaps values at offsets a and b.voidtrim()Final clean up of storage.voidtrimAfterNgram(int ngramOrder, long size) Clear storage after an n-gram order is completeprotected booleanMethods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitMethods inherited from interface edu.berkeley.nlp.lm.values.ProbBackoffValueContainer
getSuffixOffsetMethods inherited from interface edu.berkeley.nlp.lm.values.ValueContainer
add, clearStorageForOrder, numValueBits, setMap, setSizeAtLeast, storeSuffixoffsets, trimAfterNgram
-
Field Details
-
valueRanks
-
storeSuffixIndexes
protected final boolean storeSuffixIndexes -
valueCoder
-
valueRadix
protected final int valueRadix -
valueWidth
protected int valueWidth -
defaultValRank
protected final int defaultValRank- See Also:
-
numNgramsForEachOrder
protected final long[] numNgramsForEachOrder -
suffixBitsForOrder
protected final int[] suffixBitsForOrder -
useMapValueArray
protected boolean useMapValueArray
-
-
Constructor Details
-
UncompressedProbBackoffValueContainer
public UncompressedProbBackoffValueContainer(LongToIntHashMap countCounter, int valueRadix, boolean storePrefixes, long[] numNgramsForEachOrder) -
UncompressedProbBackoffValueContainer
public UncompressedProbBackoffValueContainer(int valueRadix, boolean storePrefixIndexes, long[] numNgramsForEachOrder, long[] probsAndBackoffsForRank, LongToIntHashMap countIndexer, int wordWidth) - Parameters:
valueRadix-storePrefixIndexes-probsAndBackoffsForRank-maxNgramOrder-hasBackoffValIndexer-noBackoffValIndexer-probsForRank-hasBackoffValIndexer-
-
-
Method Details
-
createFreshValues
Description copied from interface:ValueContainerCreates a fresh value container for copying purposes.- Specified by:
createFreshValuesin interfaceValueContainer<ProbBackoffPair>- Returns:
-
getProb
public final float getProb(int ngramOrder, long index) - Specified by:
getProbin interfaceProbBackoffValueContainer
-
getInternalVal
public final long getInternalVal(int ngramOrder, long index) -
getProb
-
getFromOffset
Description copied from interface:ValueContainerGets the value living at a particular offset.- Specified by:
getFromOffsetin interfaceValueContainer<ProbBackoffPair>- Parameters:
index-ngramOrder-
-
getBackoff
public final float getBackoff(int ngramOrder, long index) - Specified by:
getBackoffin interfaceProbBackoffValueContainer
-
getBackoff
-
getDefaultVal
-
getFromRank
-
getScratchValue
Description copied from interface:ValueContainerCreates a fresh value of object (useful for passing as an output parameter)- Specified by:
getScratchValuein interfaceProbBackoffValueContainer- Specified by:
getScratchValuein interfaceValueContainer<ProbBackoffPair>- Returns:
-
setFromOtherValues
Description copied from interface:ValueContainerDestructively sets internal storage from another object.- Specified by:
setFromOtherValuesin interfaceValueContainer<ProbBackoffPair>- Parameters:
o-
-
trim
public void trim()Description copied from interface:ValueContainerFinal clean up of storage.- Specified by:
trimin interfaceValueContainer<ProbBackoffPair>
-
getCountRank
protected long getCountRank(long val) -
useValueStoringArray
protected boolean useValueStoringArray() -
setMap
Description copied from interface:ValueContainerInitializes a value container with the map that contains it- Specified by:
setMapin interfaceValueContainer<V extends LongRepresentable<V>>
-
swap
public void swap(long a, long b, int ngramOrder) Description copied from interface:CompressibleValueContainerSwaps values at offsets a and b.- Specified by:
swapin interfaceCompressibleValueContainer<V extends LongRepresentable<V>>- Parameters:
a-b-ngramOrder-
-
add
public boolean add(int[] ngram, int startPos, int endPos, int ngramOrder, long offset, long prefixOffset, int word, ProbBackoffPair val_, long suffixOffset, boolean ngramIsNew) Description copied from interface:ValueContainerAdds a new value at the specified offset.- Specified by:
addin interfaceValueContainer<V extends LongRepresentable<V>>- Parameters:
ngramOrder- As always, ngramOrder is 0-based (0=unigram)offset-prefixOffset-word-val_-suffixOffset-- Returns:
- Whether or not the add was successful
-
getCompressed
Description copied from interface:CompressibleValueContainerCompresses the value at the given offset into a list of bits.- Specified by:
getCompressedin interfaceCompressibleValueContainer<V extends LongRepresentable<V>>- Parameters:
offset-ngramOrder-- Returns:
-
decompress
public void decompress(BitStream bits, int ngramOrder, boolean justConsume, ProbBackoffPair outputVal) Description copied from interface:CompressibleValueContainerReads and decompresses from the bit stream bits.- Specified by:
decompressin interfaceCompressibleValueContainer<V extends LongRepresentable<V>>- Parameters:
bits-ngramOrder-justConsume- If true, nothing is returned, and the function simply consumes the appropriate number of bits from the BitStream.
-
setSizeAtLeast
public void setSizeAtLeast(long size, int ngramOrder) Description copied from interface:ValueContainerSets internal storage for size for a particular n-gram order- Specified by:
setSizeAtLeastin interfaceValueContainer<V extends LongRepresentable<V>>- Parameters:
size-ngramOrder-
-
getSuffixOffset
public long getSuffixOffset(long index, int ngramOrder) -
getSuffixOffsetFromInternalVal
protected int getSuffixOffsetFromInternalVal(long internalVal) - Parameters:
internalVal-- Returns:
-
getRank
protected long getRank(int ngramOrder, long offset) -
clearStorageAfterCompression
public void clearStorageAfterCompression(int ngramOrder) - Specified by:
clearStorageAfterCompressionin interfaceCompressibleValueContainer<V extends LongRepresentable<V>>
-
trimAfterNgram
public void trimAfterNgram(int ngramOrder, long size) Description copied from interface:ValueContainerClear storage after an n-gram order is complete- Specified by:
trimAfterNgramin interfaceValueContainer<V extends LongRepresentable<V>>- Parameters:
ngramOrder-size-
-
clearStorageForOrder
public void clearStorageForOrder(int ngramOrder) - Specified by:
clearStorageForOrderin interfaceValueContainer<V extends LongRepresentable<V>>
-
storeSuffixoffsets
public boolean storeSuffixoffsets()- Specified by:
storeSuffixoffsetsin interfaceValueContainer<V extends LongRepresentable<V>>
-
numValueBits
public int numValueBits(int ngramOrder) - Specified by:
numValueBitsin interfaceValueContainer<V extends LongRepresentable<V>>
-