Satoshi MailBeta Open in Satoshi Mail

Re: Linux build

Private email with Martti Malmi (sirius-m) · 10 messages · from 29 October 2009 · with Satoshi Nakamoto, mmalmi@cc.hut.fi

Read in Satoshi Mail Archive source

Satoshi Nakamoto to mmalmi@cc.hut.fi · · original
The easy solution I took was to look at the wxWidgets source code and see how they did it. They just mapped it to wxMutex on non-MSW, which does have TryEnter, so that mapped in perfectly. I checked in all my backlog of changes to SVN, including the overhaul of CCriticalSection in util.h and OpenSSL's mutex callback in util.cpp to do everything with wxWidgets when not on Windows. If we get it working on Linux, I'll run my test suite against it here off-network first, then we can give an unreleased build to LibertyStandard to test for a while before going public.
mmalmi@cc.hut.fi wrote:
> I'll convert the CriticalSection code to wxCriticalSection and upload > it to SVN (it's a little tricky). I don't know what to do for > TryEnterCriticalSection though. I think I'm almost ready to check > everything in. Would the Boost mutex be of any help here? http://www.boost.org/doc/libs/1_40_0/doc/html/thread/synchronization.html#thread.synchronization.mutex_concepts
Satoshi Nakamoto to Martti Malmi <mmalmi@cc.hut.fi> · · original
I fixed some non-portable stuff I came across: QueryPerformanceCounter %I64d in printf format strings Sleep CheckDiskSpace If there's any other unportable stuff you know of I should fix, let me know. I think I'll move debug.log and db.log into the same directory as the data files (%appdata%\Bitcoin), rather than whatever the current directory happens to be.
mmalmi@cc.hut.fi to Satoshi Nakamoto <satoshin@gmx.com> · · original
I made an #ifdef to replace QueryPerformanceCounter with Linux's gettimeofday in util.h. Some Unicode/ANSI errors were resolved without code changes when I updated to wxWidgets 2.9. The only compile error I'm getting in Linux at the moment is from heapchk() in util.h.
I fixed some non-portable stuff I came across: QueryPerformanceCounter %I64d in printf format strings Sleep CheckDiskSpace If there's any other unportable stuff you know of I should fix, let me know. I think I'll move debug.log and db.log into the same directory as the data files (%appdata%\Bitcoin), rather than whatever the current directory happens to be.
Satoshi Nakamoto to mmalmi@cc.hut.fi · · original
heapchk() is just a MSVCRT debugging thing that's not being used. It can be a no-op on Linux. OpenSSL automatically uses /dev/urandom to seed on Linux, so RandAddSeedPerfmon can also be a no-op. Don't let it connect to the network before we've tested it thoroughly off-net. If you have two computers, unplug the internet and use "bitcoin -connect=<ip>" to connect to each other, one windows and one linux. -connect will allow you to connect to non-routable addresses like 192.168.x.x. We don't want to reflect badly on the reliability of the network if it throws off some malformed crud we hadn't thought to check for yet, or discovers something else anti-social to do on the network. I have time that I can do some testing when you've got something buildable to test. I can include it in the stress test I'm currently running on the changes so far.
mmalmi@cc.hut.fi wrote:
I made an #ifdef to replace QueryPerformanceCounter with Linux's gettimeofday in util.h. Some Unicode/ANSI errors were resolved without code changes when I updated to wxWidgets 2.9. The only compile error I'm getting in Linux at the moment is from heapchk() in util.h. > I fixed some non-portable stuff I came across: > QueryPerformanceCounter > %I64d in printf format strings > Sleep > CheckDiskSpace > > If there's any other unportable stuff you know of I should fix, let me > know. > > I think I'll move debug.log and db.log into the same directory as the > data files (%appdata%\Bitcoin), rather than whatever the current > directory happens to be.
mmalmi@cc.hut.fi to Satoshi Nakamoto <satoshin@gmx.com> · · original
I uploaded what I've ported so far to the svn/branches. Util, script, db and the headers compile fully now and net.cpp partially, so there's still work to do. _beginthread doesn't have a direct Linux equivalent, so I used Boost threads instead. I couldn't get connected using the Tor SOCKS proxy. That might be because of the Freenode Tor policy which requires connecting to their hidden service: http://freenode.net/irc_servers.shtml#tor
heapchk() is just a MSVCRT debugging thing that's not being used. It can be a no-op on Linux. OpenSSL automatically uses /dev/urandom to seed on Linux, so RandAddSeedPerfmon can also be a no-op. Don't let it connect to the network before we've tested it thoroughly off-net. If you have two computers, unplug the internet and use "bitcoin -connect=<ip>" to connect to each other, one windows and one linux. -connect will allow you to connect to non-routable addresses like 192.168.x.x. We don't want to reflect badly on the reliability of the network if it throws off some malformed crud we hadn't thought to check for yet, or discovers something else anti-social to do on the network. I have time that I can do some testing when you've got something buildable to test. I can include it in the stress test I'm currently running on the changes so far. mmalmi@cc.hut.fi wrote: > I made an #ifdef to replace QueryPerformanceCounter with Linux's > gettimeofday in util.h. Some Unicode/ANSI errors were resolved > without code changes when I updated to wxWidgets 2.9. The only > compile error I'm getting in Linux at the moment is from heapchk() > in util.h. > >> I fixed some non-portable stuff I came across: >> QueryPerformanceCounter >> %I64d in printf format strings >> Sleep >> CheckDiskSpace >> >> If there's any other unportable stuff you know of I should fix, >> let me know. >> >> I think I'll move debug.log and db.log into the same directory as the >> data files (%appdata%\Bitcoin), rather than whatever the current >> directory happens to be. > > >
Satoshi Nakamoto to mmalmi@cc.hut.fi · · original
It was almost there. I fixed a few things and got it to finish compiling but I don't know the system libraries to link to so there's undefined references galore. I changed the makefile to look for things under /usr/local and in their default "make install" locations. I wrote what I did and switches I used in build-unix.txt. I'm currently using wxWidgets 2.8.9 for now because it's the same version as on Windows and I don't want to wonder if there's version change issues at the same time as platform change. 2.8.10 or 2.9.0 are probably fine though. I went with the single-library compile of wxWidgets since we're linking to almost every library anyway. I added xpm files, which is what they use everywhere else but Windows instead of RC files. They're clever C files that define graphics in static arrays. The bitcoin icon has 5 different versions but I couldn't figure out how that works in xpm so I only put the biggest one. Maybe on GTK it scales it for you. I don't know if these are right or what, but they compile.
mmalmi@cc.hut.fi wrote:
I uploaded what I've ported so far to the svn/branches. Util, script, db and the headers compile fully now and net.cpp partially, so there's still work to do. _beginthread doesn't have a direct Linux equivalent, so I used Boost threads instead.
Satoshi Nakamoto to mmalmi@cc.hut.fi · · original
Just letting you know I'm still working on the Linux build so we don't duplicate work. I got it linked and ran it and working through runtime issues like getting it switched to load bitmaps from xpm instead of resources. There are debian packages available for some of the dependencies instead of having to compile them ourselves: apt-get install build-essential apt-get install libgtk2.0-dev apt-get install libssl-dev I need to see if Berkeley DB or Boost have packages. We'll shared-link OpenSSL, I'm pretty sure it's always preinstalled on Linux. GTK has to be shared linked. I'm not completely sure if it's preinstalled by default.
mmalmi@cc.hut.fi to Satoshi Nakamoto <satoshin@gmx.com> · · original
Besides the IRC part, how did your test of proxy go? Since you've been connected before, your addr.dat contains known node addresses, but without IRC to know which ones are online, it takes a long time to find them. There are normally 1 to 3 other nodes besides you that can accept incoming connections, and existing nodes that already know you would eventually connect to you. How many connections did you get, and how long did it take? I guess to know whether it successfully connected outbound through TOR you'd need to search debug.log for "connected".
Enabling the proxy setting and restarting Bitcoin I got the first connections in less than a minute and ultimately even 8 connections. I wonder if they're all really through TOR. Netstat shows only 2 connections to localhost:9050 and 7 connections from local port 8333 to elsewhere. (Some of the shown connections may be already disconnected ones.) For some reason there's no debug.log in the folder where I'm running it.
If some nodes that accept incoming connects were willing to have their IP coded into the program, it could seed automatically. Or some IP seed addresses posted on a Wiki page with the instructions.
The wiki page sounds like a good and quickly applicable solution. I could keep my ip updated there and we could ask others to do the same. When the Linux build works, it's easier to set up nodes on servers that are online most of the time and have a static IP. A static ip list shipped with Bitcoin and a peer exchange protocol would be cool. That way there'd be no need for an IRC server.
Just letting you know I'm still working on the Linux build so we don't duplicate work. I got it linked and ran it and working through runtime issues like getting it switched to load bitmaps from xpm instead of resources.
Ok. I didn't get it linked on the first attempt, but I didn't look further into the dependencies yet.
Satoshi Nakamoto to Martti Malmi <mmalmi@cc.hut.fi> · · original
I merged the linux changes into the main trunk on SVN. It compiles and runs now. I think all the problems are in the UI. The menus quickly quit working and it doesn't repaint when it's supposed to unless I resize it, and the UI is getting some segfaults. Shouldn't be too hard to debug with gdb. I haven't tested if it plays nice with other nodes yet so keep it off-net. build-unix.txt and makefile.unix added
Satoshi Nakamoto to Martti Malmi <mmalmi@cc.hut.fi> · · original
It works reliably on Linux now, except if it uses wxMessageBox() outside the GUI thread, it'll crash because non-GUI threads can't open a window on Linux. I haven't got to fixing that yet. I've been running my stress test on it and it's functioning normally. Most of wxWidgets is not thread-safe to use in threads other than the UI thread, but as a rule of thumb on Windows anything not UI related is OK. It turns out its more thread-unsafe on GTK. I replaced a bunch of stuff at once so I don't know if it was just one thing (probably Repaint), but I have to assume even any wx function that uses wxString is not safe to use outside the UI thread. So dang, there goes all the nice wxWidgets portability support functions. I left a few simple things like wxThread::GetCPUCount() that I checked the source and it's all numerical, and wxMutex has to be safe or it'd be useless. There's an issue that if you exit and run it again right away, it can't bind port 8333. The port frees up after about a minute. Unless I'm missing something, I am closing the socket before exit, so I don't know what else I can do. Maybe this is just something about Linux that it takes a minute to free up a port you had bound. Possibly a security feature so some trojan doesn't kill the web server and quickly jump into its place and pick up all the client retries. Still gotta figure out how to do the xpm version of the icon correctly. I wonder if the database dat files are interchangeable with Windows.