The complexity depends on requirements of the codec used for the transport format. Some codecs can be treated as unstructured octet streams as they either do not need to structure their data in a more complex way or they organize the information into chunks using their own means, while others use the means provided by a file format (like ogg) or by a transport protocol (like RTP) to carry the information about the structure.
Codecs from the first group are simpler to handle as you can use just tshark -r file_name -Y "rtp.ssrc == your_ssrc_value" -T fields -e rtp.payload
to export the payload as lines of colon-separated hex values which you can then convert to a binary byte stream using a simple script. Unless the playback software supports processing of raw data (where you provide the information about codec and its settings manually), the script has to provide some header before the actual data so that the playing application knows what codec with what settings to use to interpret the contents.
There used to be a direct export of binary octet stream from RTP in Wireshark 1.x but as of current, it is not available in Wireshark 2.x
Codecs from the second group need that the script or application packs the same hex lines output from tshark into some file format which supports the information about data chunk borders, providing other metadata where necessary. Different codecs are supported in conjuction with different file formats.