Add Unicode conversion functions to convert between Windows native UTF-16LE
encoding to UTF-8 and back.
To support repositories with legacy-encoded file names, the UTF-8 to UTF-16
conversion function tries to create valid, unique file names even for
invalid UTF-8 byte sequences, so that these repositories can be checked out
without error.
The current implementation leaves invalid UTF-8 bytes in range 0xa0 - 0xff
as is (producing printable Unicode chars \u00a0 - \u00ff, equivalent to
ISO-8859-1), and converts 0x80 - 0x9f to hex-code (\u0080 - \u009f are
control chars).
The Windows MultiByteToWideChar API was not used as it either drops invalid
UTF-8 sequences (on Win2k/XP; producing non-unique or even empty file
names) or converts them to the replacement char \ufffd (Vista/7; causing
ERROR_INVALID_NAME in subsequent calls to file system APIs).
Signed-off-by: Karsten Blees <blees@dcon.de>
Winansi.c has many static variables that are accessed and modified from
the [v][f]printf / fputs functions overridden in the file. This may cause
multi threaded git commands that print to the console to produce corrupted
output or even crash.
Additionally, winansi.c doesn't override all functions that can be used to
print to the console (e.g. fwrite, write, fputc are missing), so that ANSI
escapes don't work properly for some git commands (e.g. git-grep).
Instead of doing ANSI emulation in just a few wrapped functions on top of
the IO API, let's plug into the IO system and take advantage of the thread
safety inherent to the IO system.
Redirect stdout and stderr to a pipe if they point to the console. A
background thread reads from the pipe, handles ANSI escape sequences and
UTF-8 to UTF-16 conversion, then writes to the console.
The pipe-based stdout and stderr replacements must be set to unbuffered, as
MSVCRT doesn't support line buffering and fully buffered streams are
inappropriate for console output.
Due to the byte-oriented pipe, ANSI escape sequences and multi-byte UTF-8
sequences can no longer be expected to arrive in one piece. Replace the
string-based ansi_emulate() with a simple stateful parser (this also fixes
colored diff hunk headers, which were broken as of commit 2efcc977).
Override isatty to return true for the pipes redirecting to the console.
Exec/spawn obtain the original console handle to pass to the next process
via winansi_get_osfhandle().
All other overrides are gone, the default stdio implementations work as
expected with the piped stdout/stderr descriptors.
Global variables are either initialized on startup (single threaded) or
exclusively modified by the background thread. Threads communicate through
the pipe, no further synchronization is necessary.
The background thread is terminated by disonnecting the pipe after flushing
the stdio and pipe buffers. This doesn't work for anonymous pipes (created
via CreatePipe), as DisconnectNamedPipe only works on the read end, which
discards remaining data. Thus we have to setup the pipe manually, with the
write end beeing the server (opened with CreateNamedPipe) and the read end
the client (opened with CreateFile).
Limitations: doesn't track reopened or duped file descriptors, i.e.:
- fdopen(1/2) returns fully buffered streams
- dup(1/2), dup2(1/2) returns normal pipe descriptors (i.e. isatty() =
false, winansi_get_osfhandle won't return the original console handle)
Currently, only the git-format-patch command uses xfdopen(xdup(1)) (see
"realstdout" in builtin/log.c), but works well with these limitations.
Many thanks to Atsushi Nakagawa <atnak@chejz.com> for suggesting and
reviewing the thread-exit-mechanism.
Signed-off-by: Karsten Blees <blees@dcon.de>
Some constants (such as LF_FACESIZE) are undefined with -DNOGDI (set in the
Makefile), and CONSOLE_FONT_INFOEX is available in MSVC, but not in MinGW.
Cast FARPROC to PGETCURRENTCONSOLEFONTEX to suppress MSVC compiler warning.
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This reverts commit df599e9612.
As of 5e9637c6 "i18n: add infrastructure for translating Git with gettext",
eval_gettext uses MinGW envsubst.exe instead of git-sh-i18n--envsubst.exe
for variable substitution. This breaks git-submodule.sh messages and tests,
as envsubst.exe doesn't support case-sensitive environment lookup (the same
is true for almost everything on Windows, including MSys and Cygwin tools).
30a615ac "Windows/i18n: rename $path to prevent clashes with $PATH" renames
the conflicting variable in git-submodule.sh, so that it works on Windows
(i.e. with case-insensitive environment, regardless of the toolset).
Revert to the documented behaviour of case-insensitive environment on
Windows.
Signed-off-by: Karsten Blees <blees@dcon.de>
GetCurrentConsoleFontEx in an atexit routine doesn't work because git
closes stdout before exit (which also closes the console handle). Check
the console font when we first encounter a non-ascii character and only
schedule the warning message to be printed at exit (warnings go to stderr,
which is not closed by git).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
MingwRT listens to _CRT_glob to decide if __getmainargs should
perform globbing, with the default being that it should.
Unfortunately, __getmainargs globbing is sub-par; for instance
patterns like "*.c" will only match c-sources in the current
directory.
Disable __getmainargs' command line wildcard expansion, so these
patterns will be left untouched, and handled by Git's superior
built-in globbing instead.
MSVC defaults to no globbing, so we don't need to do anything
in that case.
This fixes t5505 and t7810.
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
The code in the MinGW main macro is getting more and more complex, move to
a separate initialization function for readabiliy and extensibility.
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Improve the dirent implementation by removing the relics that were once
necessary to plug into the now unused MinGW runtime, in preparation for
Unicode file name support.
Move FindFirstFile to opendir, and FindClose to closedir, with the
following implications:
- DIR.dd_name is no longer needed
- chdir(one); opendir(relative); chdir(two); readdir() works as expected
(i.e. lists one/relative instead of two/relative)
- DIR.dd_handle is a valid handle for the entire lifetime of the DIR struct
- thus, all checks for dd_handle == INVALID_HANDLE_VALUE and dd_handle == 0
have been removed
- the special case that the directory has been fully read (which was
previously explicitly tracked with dd_handle == INVALID_HANDLE_VALUE &&
dd_stat != 0) is now handled implicitly by the FindNextFile error
handling code (if a client continues to call readdir after receiving
NULL, FindNextFile will continue to fail with ERROR_NO_MORE_FILES, to
the same effect)
- extracting dirent data from WIN32_FIND_DATA is needed in two places, so
moved to its own method
- GetFileAttributes is no longer needed. The same information can be
obtained from the FindFirstFile error code, which is ERROR_DIRECTORY if
the name is NOT a directory (-> ENOTDIR), otherwise we can use
err_win_to_posix (e.g. ERROR_PATH_NOT_FOUND -> ENOENT). The
ERROR_DIRECTORY case could be fixed in err_win_to_posix, but this
probably breaks other functionality.
Removes the ERROR_NO_MORE_FILES check after FindFirstFile (this was
fortunately a NOOP (searching for '*' always finds '.' and '..'),
otherwise the subsequent code would have copied data from an uninitialized
buffer).
Changes malloc to git support function xmalloc, so opendir will die() if
out of memory, rather than failing with ENOMEM and letting git work on
incomplete directory listings (error handling in dir.c is quite sparse).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Git-compat-util.h is two dirs up, and already includes <dirent.h> (which
is the same as "dirent.h" due to -Icompat/win32 in the Makefile).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
FILENAME_MAX and MAX_PATH are both 260 on Windows, however, MAX_PATH is
used throughout the other Win32 code in Git, and also defines the length
of file name buffers in the Win32 API (e.g. WIN32_FIND_DATA.cFileName,
from which we're copying the dirent data).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Remove the union around dirent.d_type and the unused dirent.d_reclen member
(which was necessary for compatibility with the MinGW dirent runtime, which
is no longer used).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
There are no proper inodes on Windows, so remove dirent.d_ino and #define
NO_D_INO_IN_DIRENT in the Makefile (this skips e.g. an ineffective qsort in
fsck.c).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Unicode console output won't display correctly with default settings
because the default console font ("Terminal") only supports the system's
OEM charset. Unfortunately, this is a user specific setting, so it cannot
be easily fixed by e.g. some registry tricks in the setup program.
This change prints a warning on exit if console output contained non-ascii
characters and the console font is supposedly not a TrueType font (which
usually have decent Unicode support).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
GetStdHandle(STD_OUTPUT_HANDLE) doesn't work for stderr if stdout is
redirected. Use _get_osfhandle of the FILE* instead.
_isatty() is true for all character devices (including parallel and serial
ports). Check return value of GetConsoleScreenBufferInfo instead to
reliably detect console handles (also don't initialize internal state from
an uninitialized CONSOLE_SCREEN_BUFFER_INFO structure if the function
fails).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
WriteConsoleW seems to be the only way to reliably print unicode to the
console (without weird code page conversions).
Also redirects vfprintf to the winansi.c version.
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Git requires the TERM environment variable to be set for all color*
settings. Simulate the TERM variable if it is not set (default on Windows).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This happens to shut up t7602 on Windows which would otherwise take
the different line endings for a sign that the merge failed.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This happens only when the corresponding commits are not exported in
the current fast-export run. This can happen either when the relevant
commit is already marked, or when the commit is explicitly marked
as UNINTERESTING with a negative ref by another argument.
This breaks fast-export basec remote helpers.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
On msysGit creating the post-rewrite.args file using 'echo' has different
line endings from the expected comparison. Using perl normalizes the line
endings for each generated file.
Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net>
Git on Windows was made aware of the fact that sometimes a file may be
used by another process and so an operation may fail but the user might
be able to fix it and is asking for confirmation whether it should
retry.
This is implemented in a way that git only asks in case stdin and stderr
are attached to a tty. Unfortunately this seems to be misdetected
sometimes causing the testsuite to hang when git is waiting for a user
answer.
This patch works around the situation.
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net>
This is really a problem with shell scripts being called on msysGit,
but there are more important bugs to fix for the moment.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Along the lines of 05d0e3b and f33946d, use cat instead of echo to avoid
line ending mismatches in the test result of "am empty-file does not
infloop" which make the test fail.
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Again, avoid using echo (which issues DOS line endings on msysGit) to not mix
with Unix line-endings issued by git built-ins, even if this is at the cost of
calling an external executable (cat) instead of a shell built-in (echo).
Incidentally, this makes grep -I respect the "binary" attribute (actually,
the "-text" attribute, but "binary" implies that).
Since the attributes are not thread-safe, we now need to switch off
threading if -I was passed.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Gitweb is not exactly what you would call server-friendly, so let's
offload one more task onto the client.
To enable this, put something like this into your gitweb_config.perl:
$feature{'syntaxhighlighter_js'}{'default'} = [{
url => '/SyntaxHighlighter/',
style => 'Django',
theme => 'FadeToGrey'
}];
and clone git://github.com/alexgorbatchev/SyntaxHighlighter into the
directory you specified via the 'url' parameter.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It appears that `pwd` returns the POSIX-style or the DOS-style path
depending which style the previous `cd` used. To normalize, enforce `pwd
-W` in scripts.
From the original e-mail exchange:
On Thu, Mar 22, 2012 at 11:13:37AM +0100, Sebastian Schuberth wrote:
> On Wed, Mar 21, 2012 at 22:21, Johannes Sixt <j6t@kdbg.org> wrote:
>
> > I build git and run its tests outside the msysgit environment. Does that
> > explain the difference? (And I use CMD.)
>
> It does not make a difference for me. I started cmd.exe at
> c:\msysgit\git\t, added c:\msysgit\bin temporarily to PATH, and ran
> "sh t5526-fetch-submodules.sh -i -v", and the test still fails.
Yes it probably does. Johannes said that he runs the tests outside of
the msysgit folder. That way there is only one path the submodule script
gets reported and not two like '/c/msysgit/git' and '/git'.
That would explain to me why it is passing.
I am afraid that the only solution is to patch msys itself to report the
long absolute path when passing window style paths to cd. Currently when
I do
cd c:/msysgit/git
I will end up in '/git' instead of the long path.
I found that there is a -W option to pwd in msys bash which makes it
always return the real windows path. A normalization in that direction
is unique and thus might be more robust. Have a look at the attached
patch. With this at least t5526 passes. I was not able to run the whole
testsuite properly at the moment. I can have a look at that tomorrow.
What do you think?
Cheers Heiko
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows XP3 in git bash
git clone git@github.com:octocat/Spoon-Knife.git
cd Spoon-Knife
git gui
menu Remote\Fetch from\origin
error: cannot spawn git: No such file or directory
error: could not run rev-list
if u run
git fetch --all
it worked normal in git bash or gitgui tools
In second version CreateProcess get 'C:\Git\libexec\git-core/git.exe' in
first version - C:/Git/libexec/git-core/git.exe and not executes (unix
slashes)
after fixing C:\Git\libexec\git-core\git.exe or
C:/Git/libexec/git-core\git.exe it works normal
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows, we would like to be able to have a default http.sslCAinfo
that points to an MSys path (i.e. relative to the installation root of
Git). As Git is a MinGW program, it has to handle the conversion
of the MSys path into a MinGW32 path itself.
Since system_path() considers paths starting with '/' as absolute, we
have to convince it to make a Windows path by stripping the leading
slash.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In git-send-email.perl, here are two checks to determine if
$smtp_server is an absolute path (so it'll be treated as a mailer) or
not (so it'll be treated as a hostname). The one that handles actual
mail processing has been taught to recognize Windows pathnames by
commit 33b2e81f.
The other check is just to tell the user what happened, so it's far
less important, but the current state is that we will still claim to
the user that c:/foo/bar is a server. =) This makes the second check
consistent with the first.
Signed-off-by: bert Dvornik <dvornik+git@gmail.com>
For a long time, this developer thought that Git's insistence that
pushing into the current branch is evil was completely merited.
Just for fun, the original patch tried to show people that Git is right
there, and that it causes more trouble than it does good when Git allows
you to try to update the working tree for fast-forwards, or to detach the
HEAD, depending on some config settings.
Surprisingly, the opposite was shown.
So here is the support for two new options you can give the config
variable receive.denyCurrentBranch:
'updateInstead':
Try to merge the working tree with the new tip of the branch
(which can lead to really horrible merge conflicts).
'detachInstead':
Detach the HEAD, thereby avoiding a disagreement between the
HEAD and the index (as well as the working tree), possibly
leaving the local user wondering how on earth her HEAD became
so detached.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
