org.jwat.warc

Class WarcWriterCompressed

java.lang.Object

org.jwat.warc.WarcWriter

org.jwat.warc.WarcWriterCompressed

All Implemented Interfaces:

Closeable, AutoCloseable

public class WarcWriterCompressed
extends WarcWriter

WARC Writer implementation for writing GZip compressed files. Use WarcWriterFactory to get an instance of this class.

Author:

nicl

Field Summary

Fields

Modifier and Type	Field and Description
protected GzipEntry	entry Current GZip entry.
protected GzipWriter	writer GZip Writer used.

Fields inherited from class org.jwat.warc.WarcWriter

bExceptionOnContentLengthMismatch, diagnostics, fieldParsers, header, headerContentLength, out, payloadWrittenTotal, S_HEADER_WRITTEN, S_INIT, S_PAYLOAD_WRITTEN, S_RECORD_CLOSED, state, stream_copy_buffer, uriProfile, warcDateFormat, warcTargetUriProfile

Method Summary

Modifier and Type	Method and Description
void	close() Close WARC writer and free its resources.
void	closeRecord() Close the WARC record in an implementation specific way.
boolean	isCompressed() Is this writer producing compressed output.
long	streamPayload(InputStream in) Stream the content of an input stream to the payload content.
byte[]	writeHeader(WarcRecord record) Write a WARC header to the WARC output stream.
long	writePayload(byte[] b) Append the content of a byte array to the payload content.
long	writePayload(byte[] b, int offset, int len) Append the partial content of a byte array to the payload content.
void	writeRawHeader(byte[] header_bytes, Long contentLength) Write a raw WARC header to the WARC output stream.

Methods inherited from class org.jwat.warc.WarcWriter

closeRecord_impl, exceptionOnContentLengthMismatch, getUriProfile, getWarcTargetUriProfile, init, setExceptionOnContentLengthMismatch, setUriProfile, setWarcTargetUriProfile, writeHeader_impl

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

writer

protected GzipWriter writer

GZip Writer used.

entry

protected GzipEntry entry

Current GZip entry.

Method Detail

isCompressed

public boolean isCompressed()

Description copied from class: WarcWriter

Is this writer producing compressed output.

Specified by:

isCompressed in class WarcWriter

Returns:

boolean indicating whether compressed output is produced

close

public void close()
           throws IOException

Description copied from class: WarcWriter

Close WARC writer and free its resources.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in class WarcWriter

Throws:

IOException - if an i/o exception occurs while closing the writer

closeRecord

public void closeRecord()
                 throws IOException

Description copied from class: WarcWriter

Close the WARC record in an implementation specific way.

Specified by:

closeRecord in class WarcWriter

Throws:

IOException - if an i/o exception occurs while closing the record

writeRawHeader

public void writeRawHeader(byte[] header_bytes,
                           Long contentLength)
                    throws IOException

Description copied from class: WarcWriter

Write a raw WARC header to the WARC output stream. Closes any previously written record that has not been closed prior to this call. Errors and warnings are reported on the writers diagnostics object.

Overrides:

writeRawHeader in class WarcWriter

Parameters:

header_bytes - raw WARC header to output

contentLength - the expected content-length to be written and validated

Throws:

IOException - if an i/o exception occurs while writing header data

writeHeader

public byte[] writeHeader(WarcRecord record)
                   throws IOException

Description copied from class: WarcWriter

Write a WARC header to the WARC output stream. Errors and warnings are reported on the records diagnostics object.

Specified by:

writeHeader in class WarcWriter

Parameters:

record - WARC record to output

Returns:

byte array version of header as it was written

Throws:

IOException - if an i/o exception occurs while writing header data

streamPayload

public long streamPayload(InputStream in)
                   throws IOException

Description copied from class: WarcWriter

Stream the content of an input stream to the payload content.

Overrides:

streamPayload in class WarcWriter

Parameters:

in - input stream containing payload data

Returns:

number of bytes written during method invocation

Throws:

IOException - if an i/o exception occurs while writing payload data

writePayload

public long writePayload(byte[] b)
                  throws IOException

Description copied from class: WarcWriter

Append the content of a byte array to the payload content.

Overrides:

writePayload in class WarcWriter

Parameters:

b - byte array with data to be written

Returns:

number of bytes written during method invocation

Throws:

IOException - if an i/o exception occurs while writing payload data

writePayload

public long writePayload(byte[] b,
                         int offset,
                         int len)
                  throws IOException

Description copied from class: WarcWriter

Append the partial content of a byte array to the payload content.

Overrides:

writePayload in class WarcWriter

Parameters:

b - byte array with partial data to be written

offset - offset to data to be written

len - length of data to be written

Returns:

number of bytes written during method invocation

Throws:

IOException - if an i/o exception occurs while writing payload data