Add the functionality back in for xterm compatibility, but do not expose the
capability in st.info (yet).

Some notes:

It was reverted because it caused some issues with ncurses in some
configurations, namely when using BSD padding (--enable-bsdpad, BSD_TPUTS) in
ncurses it caused issues with repeating digits.

A fix has been upstreamed in ncurses since snapshot 20200523. The fix is also
backported to OpenBSD -current.

This reverts commit e8392b282c.

There is currently a bug in older ncurses versions (like on OpenBSD) where a
fix for a bug with REP is not backported yet. Most likely in tty/tty_update.c:

Noticed while using lynx (which uses ncurses/curses).
To reproduce using lynx: echo "Z0000000" | lynx -stdin

or using the program:

int
main(void)
{
	WINDOW *win;
	win = initscr();

	printw("Z0000000");

	refresh();

	sleep(5);

	return 0;
}

This prints "ZZZZZZZ" (incorrectly).

The sequence \e[Nb prints the last printed char N (more) times if it's
printable, and it's ignored after newline or other control chars.

This is Ecma-048/ANSI-X3.6 sequence and not DEC VT. It's supported by
xterm, and ncurses uses it when possible, e.g. when TERM is xterm* (and
with this commit also st*).

xterm supports only codepoints<=255, possibly due to internal limits.
We support any value/codepoint which was placed in a cell.

To test:
- tput rep 65 4 -> prints 'AAAA'
- printf "\342\225\246\033[4b" -> prints U+2566 1+4 times.

Patch by Steve Ward, thanks.

In particular on OpenBSD and on glibc wcwidth() is quite expensive.
On musl there is little difference.

Fix an issue with incorrect (partial) written sequences when libc wcwidth() ==
-1. The sequence is updated to on wcwidth(u) == -1:

	c = "\357\277\275"

but len isn't.

A way to reproduce in practise:

* st -o dump.txt
* In the terminal: printf '\xcd\xb8'
- This is codepoint 888, on OpenBSD it reports wcwidth() == -1.
- Quit the terminal.
- Look in dump.txt (partial written sequence of "UTF_INVALID").

This was introduced in:

"	commit 11625c7166
	Author: czarkoff@gmail.com <czarkoff@gmail.com>
	Date:   Tue Oct 28 12:55:28 2014 +0100

	    Replace character with U+FFFD if wcwidth() is -1

	    Helpful when new Unicode codepoints are not recognized by libc."

Change:

Remove setting the sequence. If this happens to break something, another
solution could be setting len = 3 for the sequence.

exit(3) is not async-signal-safe but, _exit(2) is.
This change prevents st to crash and dump core.

Updating XIM cursor position is expensive, so only update it when cursor
position changed.

When a read operation returns 0 then it means that we arrived to the end of the
file, and new reads will return 0 unless you do some other operation such as
lseek(). This case happens with USB-232 adapters when they are unplugged.

Scroll is a program that stores all the lines of its child and be used in st as
a way of implementing scrollback.

This solution is much better than implementing the scrollback in st itself
because having a different program allows to use it in any other program
without doing modifications to those programs.

Strings which an application sends to the terminal in OSC, DCS, etc
are typically small (title, colors, etc) but one exception is OSC 52
which copies text to the clipboard, and is used for instance by tmux.

Previously st cropped these strings at 512 bytes, which for OSC 52
limited the copied text to 382 bytes (remaining buffer space before
base64). This made it less useful than it can be.

Now it's a dynamic growing buffer. It remains allocated after use,
resets to 512 when a new string starts, or leaked on exit.

Resetting/deallocating the buffer right after use (at strhandle) is
possible with some more code, however, it doesn't always end up used,
and to cover those cases too will require even more code, so resetting
only on new string is good enough for now.

STRescape holds strings in escape sequences such as OSC and DCS, and
its buffer is 512 bytes.

If the input is too big then trailing chars are ignored, but the test
was off-by-1 such that it took 510 chars instead of 511 (before a
terminating NULL is added).

Now the full size can be utilized.

Previously, base64dec checked terminating input '\0' every 4 calls to
base64dec_getc, where the latter progressed one or more chars on each
call, and could read past '\0' in the way it was used.

The input to base64dec currently comes only from OSC 52 escape seq
(copy to clipboard), and reading past '\0' or even past the buffer
boundary was easy to trigger.

Also, even if we could trust external input to be valid base64, there
are different base64 standards, and not all of them require padding
to 4 bytes blocks (using trailing '=' chars).

It didn't affect short OSC 52 strings because the buffer is initialized
to 0's, so typically it did stop within the buffer, but if the string
was trimmed to fit (the buffer is 512 bytes) then it did also read past
the end of the buffer, and the decoded suffix ended up arbitrary.

This patch makes base64dec_getc not progress past '\0', and instead
produce fake trailing padding of '='.

Additionally, at base64dec, if padding is detected at the first or
second byte of a quartet, then we identify it as invalid and abort
(a valid quartet has at least two leading non-padding bytes).

"use iswspace()/iswpunct() to find word delimiters

    this inverts the configuration logic: you no longer provide a list of
    delimiters -- all space and punctuation characters are considered
    delimiters, unless listed in extrawordchars."

Feedback from IRC and personal preference.

also print explicitly "(null)" when printf "%s" p=NULL.

noticed when exiting mutt: printf '\x1b]104\x07'

this inverts the configuration logic: you no longer provide a list of
delimiters -- all space and punctuation characters are considered
delimiters, unless listed in extrawordchars.

Features:

- Allow input methods swap with hotkey (E.g. left ctrl + left shift).
- Over-the-spot pre-editing style, pre-edit data placed over insertion point.
- Restart IME without segmentation fault.

TODO:

- Automatically pickup IME if st started before IME

And move it to the patches section.
Keeping it would force to add an exec pledge on OpenBSD, and some
people think it's bloated, so bye!

feedback from Klemens, thanks

Simplifies logic in a couple places and removes a redundant function
call.

Signed-off-by: Devin J. Pohly <djpohly@gmail.com>

Signed-off-by: Devin J. Pohly <djpohly@gmail.com>

When possible, declare functions/variables static and move struct
definitions out of headers.  In order to allow utf8decode to become
internal, use codepoint for DECSCUSR extension directly.

Signed-off-by: Devin J. Pohly <djpohly@gmail.com>

Prefer passing arguments to declaring external global variables.  The
only remaining usage of extern is for config.h variables which are
needed in st.c instead of x.c (where it is now included).

Signed-off-by: Devin J. Pohly <djpohly@gmail.com>

The xinit function only needs to the rows/cols, so pass those in rather
than accessing term directly.  With a bit of arithmetic, we are able to
avoid the need for term.row and term.col in x2col, y2row, and
xdrawglyphfontspecs as well, completing the removal.

Term is now fully internal to st.c.

Signed-off-by: Devin J. Pohly <djpohly@gmail.com>

Gradually reducing x.c dependency on Term object.  Old and new cursor
glyph/position are passed to xdrawcursor.  (There may be an opportunity
to refactor further if we can unify "clear old cursor" and "draw new
cursor" functionality.)

Signed-off-by: Devin J. Pohly <djpohly@gmail.com>

Introduces three functions to encapsulate X-specific behavior:
 * xdrawline: draws a portion of a single line (used by drawregion)
 * xbegindraw: called to prepare for drawing (will be useful for e.g.
   Wayland) and returns true if drawing should happen
 * xfinishdraw: called to finish drawing (used by draw)

Signed-off-by: Devin J. Pohly <djpohly@gmail.com>