Split Text

How does the split work?

Pick the criterion (characters, words, lines or regex) and the size of each block. The text is sliced according to the size you set, and whole words are kept intact when you split by chars or words.

Every block comes numbered and with its own count. That helps when you're building Twitter/X threads, breaking long messages into SMS or processing text in chunks.

All the processing happens in your browser.

When to split a long text: character limits, organization and AI processing

Splitting a long text into smaller pieces sounds trivial until you have to do it well. The moment you try to publish a 1,200-word essay on a microblogging platform, send a multi-page memo by SMS, or feed a 200-page PDF into a language model, you hit a wall: every channel has a maximum size per message, and every algorithm has a maximum context window. A text splitter turns one big block of prose into a series of well-formed fragments that fit those constraints without losing meaning.

There are three broad reasons to split. Publishing: social networks, messaging apps and email subjects impose per-message limits, so longer pieces are broken into threads or parts. Human consumption: a wall of text is harder to read than short paragraphs. Machine consumption: language models cannot read documents of arbitrary length, so RAG pipelines and embedding generators must split material into chunks that fit the context window.

Common character limits across platforms

The standard X (formerly Twitter) post limit remains 280 characters for free accounts, while premium subscribers can publish up to 25,000. URLs always count as 23 characters regardless of their real length because X wraps them with the t.co shortener, and emojis count as two characters each. When you reply to another post, the @mention of the recipient does not count toward your 280-character budget.

SMS is more nuanced. A single SMS carries 160 characters when encoded with GSM-7 (the default Latin alphabet, 7 bits per character) and only 70 characters when encoded with UCS-2 (the Unicode set used as soon as any emoji, smart quote or non-Latin character appears). Multi-segment messages drop to 153 GSM-7 or 67 UCS-2 characters per segment, because each segment reserves bytes for a user-data header that lets the phone reassemble the message. A single emoji can turn a 160-character SMS into a three-segment UCS-2 message and triple the cost.

WhatsApp does not enforce a per-message limit you hit in normal use (technical ceiling around 65,536 characters), but the UX does: anything over a few hundred characters collapses behind a "Read more" link. Email subject lines render correctly up to roughly 60 characters on mobile. Instagram captions accept 2,200; LinkedIn posts 3,000; YouTube descriptions 5,000.

Splitting strategies: characters, words, sentences, paragraphs and markers

There are several ways to split, and the right one depends on what you plan to do with the pieces. Fixed character count is the simplest and most predictable, but will cut a word in half if you do not pad the boundary. Splitting by complete words preserves lexical integrity. Splitting by sentence preserves grammatical units and is ideal for human reading. Splitting by paragraph preserves logical sections and is the safest option for long-form prose. Splitting by a custom marker, such as a double newline, lets you control boundaries explicitly.

For threads, the typical recipe is "complete word, target 280 characters, leave 6 to 10 characters for the (1/n) counter". For SMS campaigns: "complete sentence, target 153 characters, never split a sentence across messages". For LLM ingestion, the dominant strategy today is recursive splitting at the largest meaningful boundary that still fits, which is what LangChain's RecursiveCharacterTextSplitter implements.

Best practices for threads on social media

A thread is not just a long post chopped into pieces. The reader sees each tweet in isolation, so every fragment has to make sense on its own. The first post should hook the reader and announce the thread; the last should close the argument or include a call to action. Number your posts (1/8, 2/8, etc.) so readers know how much more to expect, and keep each post under the platform limit with margin so the numbering does not push you over. Research summarised by social-media analytics tools shows threads of five to eight posts perform best on X, and the most effective individual posts sit between 71 and 100 characters.

Chunking for LLMs: chunk size and overlap

Feeding a long document to a language model is the modern reason engineers reach for a text splitter. Every LLM has a context window measured in tokens (roughly four English characters per token). You split the document into chunks, embed each, store the embeddings in a vector database, and at query time retrieve the most relevant chunks as the model context.

Two parameters dominate. chunk_size is the maximum size of each chunk, usually in characters or tokens; typical values range from 200 to 1,500 characters. chunk_overlap is the number of characters that the end of one chunk repeats at the start of the next, so a sentence that falls across a boundary is not orphaned; typical overlap is 10 to 20 percent of the chunk size. Smaller chunks give cheaper, more focused retrieval but risk losing context; larger chunks preserve context but dilute the answer with irrelevant text.

Simple split vs. RecursiveCharacterTextSplitter

The simplest algorithm splits a string by a single separator and slices the result into fixed-size buckets. This mangles prose. The RecursiveCharacterTextSplitter from LangChain tries a list of separators in order, defaulting to ["\n\n", "\n", " ", ""], and only falls back to a finer separator when a chunk is still too large. Paragraph boundaries are respected when possible, sentence boundaries when paragraphs are too big, word boundaries when sentences are too big, and arbitrary character boundaries only as a last resort. Specialised variants exist for markdown, code and HTML, each respecting the natural structure of their format.

Common mistakes when splitting text

• Cutting a word, URL or hashtag in the middle when splitting by fixed character count.
• Forgetting that emojis count as two or four bytes and silently switch SMS encoding to UCS-2, halving the per-segment limit.
• Forgetting the (1/n) counter when budgeting the per-post size, leaving you 4 to 8 characters short on the last few posts of a thread.
• Losing context between LLM chunks by setting chunk_overlap to zero.
• Splitting JSON, CSV or other structured formats with a prose splitter and breaking the syntax.
• Assuming character count equals byte count: a single emoji can be one, two, three or four bytes in UTF-8.

FAQ

How long should each chunk be for an LLM? A common starting point is 1,000 characters with 200 characters of overlap, then tune up or down based on the quality of the answers you observe.

Does my emoji count as one character or two on X? X counts each emoji as two characters toward the 280-post limit, even though the character is logically one symbol.

Why does my 160-character SMS arrive as three messages? If the text contains an emoji, a smart quote or a non-Latin character, the gateway switches encoding to UCS-2 and the per-segment limit drops from 153 to 67 characters.

Should I number every post in a thread? Yes, especially for threads of five or more posts; numbering (1/8, 2/8, ...) helps the reader navigate and signals that the thread is finite.

Can I split markdown or code with this tool? Yes, but use a custom separator that matches your structure (for example a blank line for markdown, or two newlines for code) to avoid breaking syntax.