Commit graph

50 commits

Author SHA1 Message Date
Dirk Hohndel
70c38de3a1 Cloud storage: be very strict about renames
Because of the structure of some of our files git too easily assumed that
they were renames and that confused the merge algorithm.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-08-25 10:43:27 -07:00
Dirk Hohndel
a45c5f1acf When building against libgit2 v0.23 or newer we can assume API23
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-08-25 10:43:22 -07:00
Dirk Hohndel
471af6c2fd Cloud storage: clean up after the merge
Make sure the branch is pointing at the merge commit, etc.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-08-25 10:43:16 -07:00
Dirk Hohndel
f5eb0e2bbb Cloud storage: move git merge into its own function
Just to keep things more readable.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-08-25 10:43:11 -07:00
Dirk Hohndel
4c31c01139 Cloud storage: better error message for dirty state
Tell us WHAT is wrong with the state.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-08-25 10:43:01 -07:00
Dirk Hohndel
b8575221b1 Cloud storage: use merge options to avoid conflicts
In many cases Subsurface will do something "reasonable" if we have
conflicting edits and then try to merge. GIT_MERGE_FILE_FAVOR_UNION means
that both edits will be added to the final file and then Subsurface should
quietly take one or the other - this will need quite a bit of testing.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-08-22 17:52:31 -07:00
Dirk Hohndel
fe0864be60 Cloud storage: merge local and remote changes if there is no conflict
We still punt if there are any conflicts (which are likely to occur if we
have touched dive sites in both changes). But in my testing at least for
fairly simple, non-conflicting changes this works and creates a correctly
merged tree.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-08-22 17:52:31 -07:00
Robert C. Helling
56c90efba8 Explicit cast to stop compiler warning
Signed-off-by: Robert C. Helling <helling@atdotde.de>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-08-17 21:09:22 -07:00
Dirk Hohndel
b554ed5ed6 Cloud storage: recognize our server certificate and accept it
On some platforms like Android the installed root certificates are rather
inconsistent. Same goes for older Windows machines. Instead of trying to
figure out how to get the user to install the right root certificates
(just kidding) we explicitly recognize our own server certificate and
allow that to override the validity assessment by the OS.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-07-13 11:20:18 -07:00
Dirk Hohndel
9e5e12b5e4 Cloud storage: provide more information in debug builds
It makes sense to hide git URLs from the end user in release builds, but
while developing and testing it's better to get more detailed information.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-07-13 07:11:55 -07:00
Dirk Hohndel
6811f77cdd Fix incorrect buffer length calculation
Oops, this is bad as it will lead to a certain buffer overrun.

The other change is more cosmetic - nested open comment '/*'.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-21 20:02:01 -07:00
Dirk Hohndel
ab90375c19 Use remote transport parameter instead of comparing strings
We pass it in, we might as well use it.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-19 07:31:15 -07:00
Linus Torvalds
eff3fd3261 git save: fix crash with no remote
Commit e21cae2d46 ("Cloud storage: sync the remote after save") broke
regular git saving without any remotes: it would never initialize the
"remote" pointer, and then use that uninitialized remote pointer to see
if it was a cloud storage remote that it should try to sync.

Fix it.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-16 21:40:31 -07:00
Dirk Hohndel
bd11988f74 Cloud storage: warn the user if we are not online
This isn't perfect, but at least tells the user if we weren't able to
connect and are working just off the local cache.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-16 14:09:37 -07:00
Dirk Hohndel
c593dea119 Cloud storage: use preference member instead of hard coded strings
This creates the basis to allow other backends to be used with the cloud
storage infrastructure.

So far this should all just transparently continue to work. A user would
have to manually add the cloud_base_url entry to the CloudStorage section
in their config file in order to use a different backend server.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-15 10:22:00 -07:00
Gaetan Bisson
77dae89b90 Fix compiling against libgit2-0.22.1
I have no idea whether the semantics is right, this patch was only
written so Subsurface compiles against the current stable release of
libgit2.

[Dirk Hohndel: added an extra conditional so this also compiles with the
	       curl-stream branch needed for https proxy support in
	       libgit2]

Signed-off-by: Gaetan Bisson <bisson@archlinux.org>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-14 20:49:37 -07:00
Dirk Hohndel
9ec155fad0 Cloud storage: clean up error propagation
We were falling of the end of a number of functions that were supposed to
return 0 on success or an error.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-13 20:03:20 -07:00
Dirk Hohndel
0c5ea57f1c Cloud storage: initialize local cache and remote on first use
There's no point in telling the user that the remote is empty. We need to
instead fix that and create the local cache and set things up for the
remote.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-13 18:36:08 -07:00
Dirk Hohndel
e6dce94088 Cloud storage: if there is no remote tracking branch, create it
When we first store things to the remote there won't be a matching branch
for it. And even if for some silly reason the remote branch got lost -
what's the point of telling the user that there is no remote branch? What
are they supposed to do about it. Let's just fix the problem and move on.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-13 18:36:08 -07:00
Dirk Hohndel
b413fabce7 Cloud storage: don't print confusing warning
This is no longer correct and is actively misleading. We now correctly
sync with the remote.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-13 18:24:27 -07:00
Dirk Hohndel
7b7568b1ba Cloud storage: setup proxy before cloning new repository
If we don't have a repository yet, we can't setup the proxy option before
calling into libgit2. Instead we use a callback.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-13 09:14:48 -07:00
Dirk Hohndel
abde615c1c Cloud storage: give user friendly error messages
Instead of showing the git URL and talking about failures to clone
repositories, simply tell the user what's happening with the cloud
storage.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-13 09:14:48 -07:00
Dirk Hohndel
3ad517d87a Cloud storage: check connectivity before trying to connect
With a new repository we take a different path.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-13 09:14:48 -07:00
Dirk Hohndel
9a22efb97b Move git related declarations into their own header file
Also change the name of the enum and make sure all the inner functions get
passed the remote transport information.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-13 09:14:48 -07:00
Dirk Hohndel
e21cae2d46 Cloud storage: sync the remote after save
This change once again tests if the remote can be reached. Even with a
fairly big data file and a medium speed internet connection the remote
sync is fast enough to call it nearly instantaneous. Maybe a couple of
seconds.

We may need more checks / different heuristics / warnings if the sync
didn't happen, etc. But for now this should allow more reasonable testing.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-12 13:56:23 -07:00
Dirk Hohndel
57f2c81125 Add header for Qt helpers used from C functions
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-12 10:20:57 -07:00
Dirk Hohndel
4d06e8f7b5 Cloud storage: check connection before connecting
libgit2 takes forever (a minute or so) to figure out that it can't connect
to a remote server.
So if we are using https as connection protocol, quickly check utilizing
RFCs 2324/7168 to make sure we can reach the cloud server (and not some
captive portal or something).

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-12 09:22:36 -07:00
Dirk Hohndel
5e0c546beb Cloud storage: Unset the proxy if none is configured
The settings are stored in the local cache repository - so without
resetting it a proxy would stay configured even if it was disabled in the
Subsurface preferences.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-12 09:22:35 -07:00
Dirk Hohndel
490d21806e Cloud storage: Setup http proxy for git connection
At the time of this commit support for this feature has not landed in
upstream libgit2, yet (but there is a pull request). Yet supporting this
here doesn't appear to cause any issue with older versions of libgit2,
either, so the http proxy support will simply not work when enabled and a
version of libgit2 that's too old is used.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-12 09:22:27 -07:00
Dirk Hohndel
8ad000846d Cloud storage: use our credentials when updating the remote
If we are pushing to an ssh or https based remote we need to use our
credentials.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-10 21:55:19 -07:00
Dirk Hohndel
26cbc32495 Cloud storage
Use the credential helpers earlier in the source. This way we can use them
for the push as well.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-10 21:54:42 -07:00
Dirk Hohndel
e81a67f9d3 Cloud storage: track the repository type and pass the information around
This is easier than having various parts of the code to the string
comparison on the URL.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-10 21:54:25 -07:00
Linus Torvalds
39a0ac965b git storage: actually update the remote repository if the local cache is more recent
Again, note that this currently only happens when you initially open the repository.

So if you do

    subsurface https://.../repo[myubranch]

it will start up by fetching the remote information, and updating the
local cache.  If you then download new dives, and do a save-and-exit, it
will save to the local cache, but it doesn't do the fetch at this point,
so the remote is now begind.

The *next* time you start subsurface, and load that git branch again, it
will fetch the remote, and now notice that the local cache is ahead of
it (because you downloaded new dives and saved them locally), and *then*
it will try to update the remote with the new information.

This is obviously bogus, but we will need to decide exactly how we want
to sync with the remote repository.  But now the core functionality is
there, it's just that we need some interface to say "sync now".
Especially in the face of spotty (or non-working) internet, you want a
GUI etc for this whole remote sync, rather than doing it unconditionally
and silently whenever you load the local cache initially.

With that caveat:

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-10 20:57:57 -07:00
Linus Torvalds
971f1c6674 git storage: re-organize code to make 'origin' remote more widely available
I'm going to try to update the remote if the local cache is more recent
when we fetch the data, which requires access to the remote over a wider
range of code.  This re-organizes the code so that we can free the
remote later without having to have nasty error handling.

We avoid the whole "if an error happened, free the remote and return" by
creating helper functions and freeing the remote in the caller, so that
all paths end up freeing it naturally.

NOTE! We want to try to update the remote when we save the local cache
too, so this whole "update remote when opening it" is incomplete. But
 (a) we do want to do it here as well
and
 (b) this is the easiest place to create the initial "push to remote"
     code without any new "sync with cloud" interfaces.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-10 20:57:13 -07:00
Linus Torvalds
1379f34792 git storage: actually update cached copy from remote
We used to fetch the remote information but not actually do anything
about it, except report when it wasn't up-to-date.

Now we actually update the local cached copy if the remote has changed.

The code does not try to actually merge things, so only fast-forward
updates are done, but that should be the normal case.  We might
eventually do some simple merging on our own, but I suspect manual
merging may be the safer option.

We don't currently ever update the remote repository, and only inform
users that our local repository is ahead of the remote.  Fixing that is
the next step.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-10 18:13:25 -07:00
Dirk Hohndel
2a110811cf Cloud storage: be more careful when trying to determine a user name
Since we use the email address (including the '@' sign) as the repository
name we have to be more careful when trying to pick an account name from a
git URL.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-06-09 20:54:01 -07:00
Dirk Hohndel
e84d8624bb Add strndup.h header so we can use this useful function
It's missing on Windows... we had this helper in liquivision.c but since I
used the function in git-access.c I figured I should just turn it into a
little helper.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-05-31 23:14:56 -07:00
Dirk Hohndel
5bbcc7f16d Cloud storage: first stab at implementing cloud storage
So far there is no mechanism to actually create a repository on the
server, so this only works with the two test repositories.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-05-31 22:35:50 -07:00
Dirk Hohndel
a6b6674780 Cloud storage: rethink the terminology used
Cloud storage makes more sense that remote storage - at least I assume
that more people are used to thinking about "storing things in the cloud".

Don't use PIN or passphrase, call it a password everywhere.

Don't use copy_string() to copy the password - the git credentials routine
asserts that password is not NULL, so make sure we at least have a pointer
to an empty string here.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-05-31 19:45:05 -07:00
Dirk Hohndel
97ef9d0ee8 Cloud storage: add support for https based remote
This will make it easier to access the remote storage.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-05-31 17:15:58 -07:00
Dirk Hohndel
29e7459d6b Remote storage: second attempt to fix build with older libgit2
That was embarrassing...

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-05-28 21:43:32 -07:00
Dirk Hohndel
c6e0b74452 Remote storage: turns out you need a newer libgit2
I was afraid that this would only work with a farily current git master of
libgit2 and it turns out I was right. The API was available, but the
correct type wasn't.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-05-28 16:09:29 -07:00
Dirk Hohndel
43254a70ca Remote storage: provide credentials for ssh based remotes
The key location is hard coded as this is only intended for our own
infrastructure, so we don't need multiple keys.

If the private key is protected by a passphrase, this needs to be manually
added to the preferences. As RemoteStorage->passphrase.

This has only be tested with a current version of libgit2 master, but
should compile with any previous version (the API doesn't exist in v0.19
and earlier - that's handled with appropriate #if clauses).

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-05-28 14:39:35 -07:00
Dirk Hohndel
95664af53b Add USE_LIBGIT23_API as cmake option
Just as we would expect, the libgit2 developers of course once again broke
their API. In order to compile against current master we need to remap
those APIs once again.

Simply call cmake with -DUSE_LIBGIT23_API

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-05-28 09:13:51 -07:00
Dirk Hohndel
e3215123d1 Fix parsing of git branch encoded in file name
If the folder has a trailing '/' we picked the wrong substring as branch
name.

Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-05-28 08:14:16 -07:00
Patrick Valsecchi
1c55f47662 Fix Ubuntu compilation error.
Ubuntu has git minor version 20 which has less parameters in
git-remote-fetch.

From ac9516984e4fbc787635b0dfd76649c8e43fab83 Mon Sep 17 00:00:00 2001
From: Patrick Valsecchi <patrick@thus.ch>
Date: Tue, 24 Feb 2015 08:45:21 +0100
Subject: [PATCH 2/2] Fix Ubuntu compilation error.

Ubuntu has git minor version 20 which has less parameters in
git-remote-fetch.

Signed-off-by: Patrick Valsecchi <patrick@thus.ch>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-02-24 11:41:11 +01:00
Linus Torvalds
fdd40d3c6a Remove unnecessary use of non-portable 'mkdir()'
This fixes the problem with Windows being the usual non-POSIX PoS that it is..

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-02-15 13:33:44 -08:00
Linus Torvalds
27b4d10f91 Start parsing remote git repository addresses
This doesn't really usefully work, but I'm committing it to have git
diff and history.  It also insanely hardcodes the git-repo cache
directory, because right now it's purely useful for development.

The big missing pieces are:

 - progress information
 - credential callbacks not implemented

where the first one makes the user interface horrible (long delays with
nothing visibly going on), and the second one makes ssh logins etc not
work.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-02-15 06:03:51 -08:00
Linus Torvalds
b770d0a6b7 git-access: use the new format_string helpers
It may be a bit less efficient to use a printf-style interface rather
than the explicit malloc and memcpy, but the code ends up simpler and
more readable.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-02-15 06:03:30 -08:00
Linus Torvalds
e287590e4b Start splitting out git repo helper routines
This doesn't actually change any code, but it moves the 'is_git_repo()'
function that is used by both loading and saving into a new git-access.c
file.

This is where I'll start doing remote repo syncing too. Knock wood.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Dirk Hohndel <dirk@hohndel.org>
2015-02-15 06:03:19 -08:00