Apple Breaks My Gitit Wiki Under Mountain Lion

So this is a pretty specific problem, and I'm just posting in case this helps anyone else who comes across this via Google.

I have a Wiki that I use on my LAN by using gitit (a Haskell wiki running on top of Git). It runs a web server, on a local port (5001), but I want to share it with other machines on my LAN, so I use Apache and some proxying. This was all working under Lion, with some effort, but when I upgraded to Mountain Lion today, I discovered that Apple really hosed up people who actually used their Apache web sharing. MANY things were broken and it's taken me some gnashing of teeth to figure it out.

First of all, they overwrote /etc/apache2/httpd.conf which was, I thought, a bit rude. Fortunately my old config file was preserved in https.conf~previous. I did not just use the old one; I pulled out the lines I had added (you always tag lines you add or change in config files with comments like PRP ADDED: (stuff) END OF PRP ADDED or PRP CHANGED FROM: (stuff) TO: (different stuff) when you change standard system config files, don't you?) Fortunately the Mountain Lion upgrade did not remove the proxy_html.conf and gitit.proxy.conf I had put in /etc/apache2/other/. Since I'm documenting this, my gitit_proxy.conf file looks like this:

# These commands will proxy /gitit/ to port 5001

ProxyRequests Off

<Proxy *>
  Order deny,allow
  Allow from all
</Proxy>

# Per the mod_proxy_html 3.1.2 sample proxy_html.conf file, this is a critical security setting

ProxyRequests Off

ProxyPass /gitit/ http://localhost:5001/

<Location /gitit/>
        # PRP: This is obsolete for mod_proxy_html versions 3.1+
        # SetOutputFilter proxy-html
        # The result is silent failure!
        # PRP: use this instead:
        ProxyHTMLEnable On
        # Note also that it is critically important that the proxy_html.conf file (default works for me)
        # is present where Apache can find it (in mod_proxy_html versions 3.1+, apparently no HTML
        # elements are changed unless they are specifically declared in a config file, so again, the
        # result is silent failure!)
        ProxyPassReverse /
        ProxyHTMLURLMap / /gitit/
        RequestHeader unset Accept-Encoding
</Location>

while my proxy.html.conf file is the standard one that came with mod_proxy_html.

Also while I'm at it, if you have problems with punctuation in Wiki URLs, I recently got some advice from the author of gitit to include xss-sanitize: no in the .conf file I use for configuring gitit, and that fixed it. For example:

[Dar Tellum: Stranger from a Distant Planet]()

was not being recognized as a Wiki link because of the colon until I changed this. Note that this might enable some sort of obscure security hole if your Wiki is facing the general public instead of behind a firewall.

Anyway, where were we? Oh yeah, Mountain Lion and the Apache web server. Apparetly after the upgrade, you can no longer turn on web sharing from the "Sharing" control panel, which was kind of a dick move on Apple's part! But you can find some instructions to turn it on here.

What I did was to enable the launch daemon with sudo defaults write /System/Library/LaunchDaemons/org.apache.httpd Disabled -bool false and then to allow me to turn it on and off more easily while testing, I installed this 3rd-party web sharing preference pane. Now personally I don't care about sharing something from my home directory or from my web root, so I didn't bother to fix the fact that accessing http://localhost/ now gives "permission denied," since I'm using a virtual server under http://localhost/gitit/, but if that bugs you, can probably find how to fix that elsewhere.

OK, there's more. I had a couple of Apache modules added to my system. They were broken. Apache wouldn't log any errors, but was failing constantly, and so the system was trying to restart it every ten seconds. I finally was able to figure out what was going wrong using apachectl configtest, which told me things like httpd: Syntax error on line 98 of /private/etc/apache2/httpd.conf: Cannot load /usr/libexec/apache2/mod_xml2enc.so into server: dlopen(/usr/libexec/apache2/mod_xml2enc.so, 10): image not found. Which meant the shared libraries were missing.

So I didn't bother to try to find if the Mountain Lion upgrade had preserved these shared libraries somewhere; I wanted to rebuild them. I had originally built them using this gentleman's advice. But under Mountain Lion this would no longer work. The commands he suggested,

$ sudo apxs -ci -I /usr/include/libxml2 mod_xml2enc.c

and

$ sudo apxs -ci -I /usr/include/libxml2 -I . mod_proxy_html.c

had several problems; the libtool commands used by apxs under Mountain Lion apparently have some new problems, so I'd get

libtool: compile: unable to infer tagged configurationlibtool: compile: specify a tag with `--tag'

I tried to fix this by changing things in /usr/share/httpd/build/config_vars.mk, but that wouldn't work, and I really didn't want to go down a libtool rabbit hole trying to understand that whole nightmare. So what I finally had to do was modify the commands that the apxs calls expanded out into. There were a couple of things I had to fix first. Although I had XCode and the command-line tools installed under Lion, when I upgraded, they were removed. This meant I had no system headers under /usr/include (the error referenced missing ctype.h). The fix was to run XCode, look under Preferences, Downloads, Components, and check for and reinstall the Command Line Tools component.

Next, note that the CC path referenced in the aforementioned config_vars.mk is broken, which looks like a Mountain Lion bug to me. So I had to change that path to correspond to the one that actually exists in the latest XCode (now installed under Applications instead of the whole Developer tree). And finally, our specific libtool used for this build needs to get --tag=CC in order to work correctly.

Inside my downloaded mod_proxy_html source directory, I used these commands:

sudo /usr/share/apr-1/build-1/libtool --silent --mode=compile --tag=CC /Applications/Xcode.app/Contents/Developer/Toolchains/XCodeDefault.xctoolchain/usr/bin/cc    -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK -I/usr/local/include -I/usr/include/apache2  -I/usr/include/apr-1   -I/usr/include/apr-1  -I/usr/include/libxml2  -c -o mod_xml2enc.lo mod_xml2enc.c && touch mod_xml2enc.slo

and

sudo /usr/share/apr-1/build-1/libtool --silent --mode=compile --tag=CC /Applications/Xcode.app/Contents/Developer/Toolchains/XCodeDefault.xctoolchain/usr/bin/cc    -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK -I/usr/local/include -I/usr/include/apache2  -I/usr/include/apr-1   -I/usr/include/apr-1  -I/usr/include/libxml2 -I.  -c -o mod_proxy_html.lo mod_proxy_html.c && sudo touch mod_proxy_html.slo

Try saying that three times fast. All I can really say is "it worked for me." Your mileage may vary.

Then I had to hand-copy the generated .so files out of the super-secret generated .libs directory and into the place Apache uses them, so:

sudo cp ./mod_proxy_html.so /usr/libexec/apache2/

and

sudo cp ./mod_xml2enc.so /usr/libexec/apache2/

And, my stuff's working again. http://localhost/gitit/ connects to my Wiki from my local machine, and also works across the network.

I hope this might save you a little time. What a mess!

Followup on Apogee Ensemble, Logic, and Skype: Making Them All Work Together

So, in this previous blog entry I talked about Apogee Ensemble, Logic, and Skype: Making Them All Work Together and included a bunch of drawings and screen-shots and what not.

After upgrading to Lion -- which was an ugly mess, in that I had a lot of plug-ins that wouldn't work right under Lion -- one problem remained, which was that I could not get bidirectional S/PDIF audio to work on the digital optical connectors. In other words, the setup that worked fine under Snow Leopard didn't seem to work under Lion.

The problem seems to be the way the Ensemble and its control panel application, Maestro, refer to the S/PDIF coaxial and optical ports. Under the previous version of Maestro, it seemed to work OK to configure the S/PDIF optical in to route to software inputs 9 and 10, so I could use 10x10 routing -- even though this wasn't actually clearly shown in Maestro -- and to route software outputs 9 and 10 to the S/PDIF optical outs.

Under Maestro 2 and the Lion drivers, this doesn't seem to work, as far as the inputs go. It seems now that the software is more clearly convinced that the coax and optical digital inputs are separate. In order to even be able to select the optical digital input for software inputs 9 and 10, you have to have the Ensemble in 18x18 mode. Like so (click the images to enlarge) -- inputs:

and outputs:

Note that what I'd really like to do is configure software inputs 11-18 as "no mapping" -- that is, I'm not using them, and similarly for the hardware outputs: I don't want any logical output channels on the coax digital out, or the three "N.A." hardware outputs (whatever that means). But whatever -- this configuration works! I can do two-way digital I/O.

However, the bigger problem is that I don't really want to run the Ensemble at 18x18 -- it uses up extra FireWire bandwidth. This means that I probably can't use my iSight camera along with the Ensemble to record a video while I'm recording audio (and I could, in the 10x10 configuration under Snow Leopard). So does it work to switch back to 10x10 after making this configuration? On the theory that I've told it what the routing should be, but I only want to route data for channels 1-10?

Nope. At least in my test, while the input to the Ensemble works fine (so, for example, I can listen via Logic to input 9/10 and hear music from my iTunes library playing from my Mac), when I try to send audio out channels 9/10 I get white noise (or what sounds mostly white noise) out of the Ensemble's digital optical S/PDIF outputs. And I can't even access the S/PDIF optical inputs or outputs in the input and output configuration windows to try to change them:

So, there you have it. A feature that was semi-working under Snow Leopard, now broken under Lion. It looks like I will have to run my Ensemble in 18x18 mode if I want to use the S/PDIF digital optical inputs and outputs.

It would be kind of nice if this were not the case -- assuming the unit is transmitting and receiving 10 channels of data, why can't it send ten arbitrary software data sources to any or multiple hardware outputs, and vice-versa? Maybe there's a reason -- I write DSP software, and sometimes for performance reasons you have to rely on a DMA, that blasts data into our out of memory in a specific ordered pattern and you can't change the pattern -- but in this case I don't think that's what is happening; I think the data is being copied out of DMA buffers into user space in arbitrary order, since you can map (say) a software output to multiple hardware outputs. And also, it worked under the previous version of MacOS X. I think the issue is just not adequately testing and supporting an edge case. Maestro has never really been the most stable and solid of applications. It seems better now, but I still managed to crash it and lock it up several times today while testing. Not what I really hope for the "flagship" audio interface for the Mac Pro, even though it is not the highest-end product in Apogee's lineup.

I spent time with Apogee's tech support chat, and although I was not able to resolve the issue while engaged in the live chat, the conversation did serve to point me in the right direction (18x18 versus 10x10 routing). Here is an (edited) transcript:

You are now chatting with 'Patricia'
Patricia: Hello! I'm an Apogee customer success representative here to assist you with your tech support needs. Let me take a look at the issue you've described, one moment please.

(Bleh -- customer "success" representative? Please...)

    
Patricia: Hi paul
Paul: Hello
Patricia: can you open apogee maestro
Patricia: and tell me your version numbers?

(Chat about software version numbers and Ensemble serial numbers, and more possibly confusing explanation of my issues, removed -- actually I was impressed with how much time the representative spent with me).

Paul: Right now it is working - I have a mic in to my Ensemble, a Logic project that processes it and routes it to 9/10 out, and an optical cable going into my Mac Pro. The sound control panel is set to Mac audio in from the optical port, and the meter reacts, and I can (for example) use this mic as a source for a Skype call this way
Paul: But it does not work in the opposite direction (I have a separate cable, the Mac audio out is set to the optical out, and I'm monitoring inputs 9/10 in Logic). I think I have Maestro configured properly although some of these settings are not that clear, like SRC Select and SRC rate. In any case they don't seem to fix it. Clock is sets to internal
Paul: Like I said this worked under Snow Leopard (with Maestro 1 and the previous drivers).
Patricia: Should it be set to 11/12?
Paul: What?
Paul: 11/12 out instead of 9/10 out?
Patricia: my apologies
Paul: The I/O allocation is set up as 10x10 -- I think that's what worked before
Patricia: on the output routing page
Patricia: ah
Patricia: then no 11/12
Paul: 9/10 are the highest numbers that show up in Logic
Patricia: that is correct if your allocation is 10x10
Paul: Was there maybe a change where the coax and S/PDIF digital outputs are separate now? I never really understood how those were counted, how it decides which ones to send/receive on?
Paul: I guess maybe that is the Optical In/ Optical Out setting. They are set to SPDIF Optical now
Patricia: In order to utilize the spidif optical you will need to use 18x18
Patricia: and use 11 and 12
Paul: But it is working one way already for the output from Logic to the mac on 9/10?
Patricia: spdif coax is 9/10
Patricia: optical is 11 and 12
Paul: Let me try that
Paul: Did that change? Because I don't think I ever had to use it at 18x18
Patricia: I am pretty sure that has always been the case

(Note that I think this is not entirely true -- or, at least, even though it may have been an unsupported configuration, I was able to get "11 and 12" optical I/O working when routing 10x10).

Paul: OK, I am changing my Logic setup... one moment
Patricia: okay
Patricia: you may have to reopen logic and re-choose the ensemble
Patricia: changing io allocations can make logic see it as a new interface
Paul: Yes I shut it down and restarted Logic
Patricia: okay
Paul: OK, Logic knows that I have more channels now, but if I output to 11/12 the Mac does not see audio on the S/PDIF in. If I output to 9/10, it does
Paul: So literally, the Sound control panel with the input set to Digital In, the meter doesn't register if Logic is sending on 11/12
Paul: As for the other direction, I'm playing music from iTunes to the Mac audio out, set to digital out, and the Ensemble front panel still shows activity on the last meter when it is playing, but I can't get anything in Logic from either 9/10 or 11/12

(Note that if I had looked at the input and output routings under the 18/18 configuration, I probably would have seen the problem at this point, but I was kind of hung up on how it had been working with a 10x10 routing).

Patricia: Well unfortunately, I don't have an ensemble here to troubleshoot with you, so I am going to have to ask you to call in for further assistance.
Patricia: 310.584.9394
Patricia: they should be able to give you more insight
Patricia: I apologize for the inconvenience
Paul: OK, well, thanks anyway
Paul: I appreciate you taking the time
Patricia: You are welcome
Patricia: they will be able to replicate your set up
Patricia: no problem!
Patricia: thank you for choosing Apogee!

Anyway, I am happy with Apogee's tech support, and could take it to the next stage and call, but since my issue is solved, I don't really have a bug to report per se. I just wish Apple (and Apogee's) audio routing was more flexible. If I could specify the exact hardware that conforms to "system audio in" and "system audio out" at the Core Audio level, with an Apple-provided GUI, this whole workaround would be largely unnecessary.