Tags: ext4

03/07/13

  01:00:00 pm, by The Dreamer   , 370 words  
Categories: Computer, BOINC, Ubuntu, FreeBSD, Virtualization, VirtualBox

VirtualBox VDI and 4K alignment

So, after my Windows 7 machine at work ate itself after a Windows Update. I decided to turn it into an Ubuntu 12.04 system.

But, so much for becoming Windows free. I was informed that I need a Windows machine to run vCenter on, because the shared Windows 2003 server that we had been using is going away.

So, I setup Windows 7 in VirtualBox...pretty simple. But, soon I was getting ext4 messages complaining about unaligned access for a certain inode. The VDI for the Windows 7 install.

I searched around on VDI and 4K alignment, and eventually found this:

HOWTO: Modify empty VDI for optimal alignment of Windows

Though reading further it seems to suggest that newer versions of VirtualBox get the alignment right for you....and 4.1.24 is much newer than 3.0.12 in the original post. (had opted for 4.1.24, since 4.2.6 is not recommended for BOINC Test4Theory....but 4.2.8 is out now, and it is recommended....so it'll be time to go and update all my machines, I guess. The only systems with 4.2.6 were FreeBSD, because the 4.1.24 will cause the 9.1 kernel to panic on VM exit. though don't know if 4.1.24 was bad in ports or not, had been 4.1.22 before upgrading to 9.1 -- but had to recompile the kernel modules after the upgrade so had first tried 4.1.24....)

So, I went through the HOW-TO to create an new emtpy VDI...and found that it is already 4K aligned. So, why was the VDI created through the 'New Wizard' in VirtualBox Manager not?

Anyways...I then booted gparted and copied my partitions over to the new VDI....and then had to boot the Win7 installer to fix the startup to get it to boot, and no more complaining.

:hmm:

Guess I'll have to remember to create VDI before creating the new VM in the future? Wonder if there are issues in other disk image formats. Wonder if ZFS would complain that my VDI isn't aligned? Wonder how to check? Though the other formats I use are either VHD or VMDK. VHD because that's what Microsoft Windows Image Backup creates, and I'm using the last one of Zen to try to recover files from.... VMDK is after an import/export of a VM or for raw zvol accesses. So, guess those aren't areas of concern for me right now.

11/05/12

  10:30:00 am, by The Dreamer   , 2100 words  
Categories: Hardware, Software, Computer, Storage, Ubuntu, FreeBSD, VirtualBox

Orac is looking strangely bare, with Zen taking over.

For a long time, I've been running a 6 drive RAID 10 array of Hitachi 5K3000 2TB drives in Orac for backuppc. This configuration got me at 5.4TB array, and somewhat better performance than when I tried a RAID6 configuration. But, eventually, I kept running out of space and the price of harddrives went up so expanding the array over time didn't happen as I had hoped. Being RAID10, the options were other concat another array, either 2 in RAID1 or 4 in RAID10 or 4 as 2 RAID1....using volume manager. Or maybe see if RAID10 would deal with having all 6 drives upgraded to 3TB, though hadn't considered the transition of 512 to 4k and how it would cope with that.

Though I did, eventually find out when I upgraded a 1.5TB RAID1 set to become a 2TB RAID1 set....going from ST31500341AS to ST2000DL003, where I contributed my experience here: http://askubuntu.com/questions/141669/creating-properly-aligned-partitions-on-a-replacement-disk/ Though it first started because one of the ST31500341AS drives had failed.

Before the failure of one of the 1.5TB drives in the above mentioned RAID1 set, I had 4 ST31500341AS in a RAID5 on old-Zen. It had been done in under RR622, under Windows, and NTFS partitions, etc. I had tried copying the data at various times, not really having anywhere that would hold the data elsewhere...but wanting to get it over to FreeBSD for recovery. While I got the rr622 driver working, and it saw that I had a single array (rather than the native driver that would see the 4 individual disks.) I couldn't get access to the data. Though it had worked when I was previously playing around with Xen (had tried copying it then....to a 2 1TB RAID0 set, but then one of the 1TB drives died....so I lost the copy, I had then replaced it with a 2TB RAID1 set....using an ST2000DL003 and an ST2000DM003, the DL being a 5900RPM drive and having a 5 year warranty...while the DM drive is a 7200RPM drive, but with only a 1 year warranty. And, turns out the 1 year is generous.

At work, I had built my FreeBSD desktop using a pair of the ST1000DM003 drives...and 3 drive failures later....it is now a pair of ST2000DL003 drives. Yeah...I was having trouble with the array, and apparently using XFS was a mistake too...because I thought it was recovering, but instead it was slowly eating the data. When I had nuked the RR622 RAID5 array, and had switched to using it as JBOD and create a RAIDZ set under FreeBSD...I found that there was nothing to copy back from the RAID1 array. D'Oh! >:XX

Though I had also copied the Microsoft WindowsImageBackup files, to see if I could mount the VHD file under VirtualBox to help in recovery. I largely had the data in bits and pieces elsewhere, it was the environment I was wanting to recreate...and Oops!Backup didn't back up that part anyways (the data I was mainly trying to migrate). The image mounted, and I could see it...but soon after Windows would try to fix it and then it would disappear....kind of like what it did on February 15th to make the original Zen go away. No idea what kind of disk rotting the Intel Matrix RAID had been doing, when it had to initialize the array again every time after a Windows crash. I've had Ubuntu crashes, but the RAID arrays remained stable...usually. While Windows & Intel RST....it was pretty much every time. I'm sure it was slowly corrupting things overtime to where things wouldn't recover, though it choose to do that after an automatic reboot for Windows updates...and the day before I left for my first Gallifrey One made things even more annoying.

Anyway with another 1.5TB drive freed up, I contemplated adding it to the RAIDZ I had made of the 4 1.5TB drives, keeping it as a hot spare, or just use it by itself -- living dangerously. I ended up with the latter for some temporary data. Because in my mind I was starting to lean to what happened next.

Pages: 1· 2

07/24/11

  11:49:00 am, by The Dreamer   , 2214 words  
Categories: Digital Photography, Hardware, Software, Computer, DVDs / NetFlix, WiFi, Storage

Morning Orac outage

Well, Orac was being somewhat unresponsive this morning.... Looks like early this morning the system stopped answering to Cacti, with the temperature of the GPU rising to a new high. Don't know what the high would've been, because there was no snmp data for a while. GPU temp did recover, but the display did not.

Ended up rebooting, needed the excuse to see external journal work....

Once backup, I switched back to internal journal...but one of maximum size rather than default size. And, then shutdown the system to extract the PCI Compact Flash card. During the reboot to see external journal work...I realized that I wasn't likely to see much performance gain from getting faster compact flash since the card itself was only UDMA/66. So the 133x CF is already faster than the card?

Probably should've fsck'd manually after the switch back....some discussion thread suggested doing this after fiddling with journal settings. But, its fsck'ng now after the boot back up. Apparently superblock had invalid external journal superblock hint? And, auto fsck failed, requiring manual fsck.

Downside of internal journal is that it can only be a max of 400MB (an improvement from the 128MB default I had before)...but that's only like 0.01% of my total space. And, there was a discussion thread that seemed to suggest a journal size of 0.2% would yield better improvement....I didn't do the whole 16GB CF card...though wonder how much that would've helped.... But 0.2% would call for a journal of ~10.9GB. Suppose I could turn off data journaling, especially since there are files that backuppc puts down that are bigger than the 400MB journal. Though I think the ordered/writeback modes usually just do metadata journaling, while I had switched it to data journaling when I was playing with external journals...but I didn't switch it back.

Wonder which of writeback or ordered would be more optimal for this FS? :??:

Full story »

Pages: 1· 2· 3

07/09/11

  09:03:13 pm, by The Dreamer   , 1290 words  
Categories: Hardware, Software, Computer, Storage

New arrays on Orac

This is an owed post (of over 3 months in the making....) I had thought of lots of things to write for this since I meant to write it, but didn't write anything. And, now its so old...that I'm just going to be really terse....

:lalala:

The lead in to this posting was Worked on Orac yesterday. So, I got those 5 2TB drives...and first tried to make it RAID5 using mdadm. And, started filling it up....though the performance didn't seem to be much better using XFS, plus the new backuppc seemed to have regressed on handling XFS.... so I heard about ZFS on Linux....as kernel modules that do the taint. And, tried out RAIDZ, with its variable striping to maybe to do better on performance. It seemed nice, but then stack dumps filling up my logs....rebooted and then did a zpool scrub, and it went to work fixing lots and lots of errors, and then a few unrecoverable errors.

Guess no ZFS, and the XFS didn't seem enough of a gain to make it different from everything else. Even though there's one really big problem with ext4. Resizing it causes corruption. Supposed to be fixed now, but before I switched to the new array, I grew the backuppc volume for 3.75TB to 3.8TB, offline...and it resulted in corruption still.

So, I was making the new backuppc volume the full size of the array. As I mulled over the situtation...I decided that I would try RAID10, sure that wouldn't get as huge a boost over the old RAID6 array of 1.5TB, but it would still be an increase...and no idea what I'm going to do when I need more space. But, first I had to get a 6th 2TB drive to do RAID10. Plus it would mean that I would need to do something else with the 1.5TB drives. One was going to have to come out. Sure it could run degraded initially (though did find that it would come back after a reboot, guess the boot making degraded arrays come online...only does it for OS filesystems. Though the boot get's stuck because it can't mount filesystems and the recovery of mounting just doesn't work still. Getting into single user is such a pain too. Though after I broke my sudoers file recently, I turned off hiddenmenu.

The hard part about making the RAID10, was figuring out the ordering of devs...so that it was mirror across the eSATA channels and stripping with in. There's only 2 channels, and 6 drives in this array...so figured that was the best way to go. Though now that I've realized that its only PCI express 1.0, not sure if that was the best way to go. Though to try the other, would call for me to get 8 drives? And, who knows what the future holds for supporting backuppc....

internal bitmap on doesn't seem to degrade the RAID10 as much as RAID5 or RAID6, so I have that turned on now.

Now there was the question of what to do with the 4 remaining drive bays and the 4 1.5TB drives. Would I got with another RAID10, a couple of RAID1...would I concat them into the LVM for growth or make them separate.

Well, on the old 1TB array was my old MMC volume, which needed more space and part of the 5x1.5TB RAID6 was a backup volume...also in need of more space. I decided that I would go with two separate RAID1s using the Seagate 1.5TB drives (I had pulled the Samsung 1.5TB to make it degraded....that drive has since been hooked up to TARDIS for local backups).

Around this time, my Roku appeared...and I went through the various attempts to map a network drive to it for local content. Settling on the HSTI Media Stick and its 1TB maximum. I settled on going with 2 1.5TB RAID1 arrays. One of them was made fully available to be the new MMC volume. And, the other...I made a 1TB volume (called TARDIS) for the HSTI Media Stick, and the rest became the new backup volume.

The old 1TB RAID1 array stayed online, until it was recently repurposed into the volume for Time Machine backups (only 931GB).

Now I've been debating playing around with external bitmaps and/or external journaling to see if I can get more performance. But, external bitmaps go away after a reboot, and that just seems to be a bug that isn't going to get fixed anytime soon. external journaling would require me to find some devices for that....so I got the idea of PCI Compact Flash adapter and some compact flash cards....I have a couple of the PCI Compact Flash adapters....and I do have a few compact flash cards around. Though when I was playing around with ReadyBoost on Zen, I found they weren't really that great on speed. And, I was going to want to get as much speed out of things as I could here....so I did some checking and now I'm waiting to get around to buying some 600x CF cards to get around to trying this. The fastest card I have available in my collection is a 133x. And, I don't recall why I bought that one. Ones earlier than that would be left overs from when I had my PowerShot S20, then an REB1200 and then a Nikon Coolpix 5700.

These days, I'm all about the SD cards for my digital cameras (Eye-Fi), though been thinking of getting an ultracompact (and wearing my holster less often)....and some that I've looked at use MicroSD cards. Don't have any extra Class 10 ones laying about though....though I do have a couple of Class 6 8GB cards that need a home (I got them in a 2for1 sale, and the need for 1 didn't happen as planned....and when I do get around to getting that device, I'm sure I'll probably cave and get a 16GB or 32GB Class 10 MicroSD card.)

Full story »

04/14/11

  07:57:00 am, by The Dreamer   , 1339 words  
Categories: Hardware, Computer, Storage, Ubuntu

Worked on Orac last night

Back on March 28th, when I last wrote about working on Orac, I mentioned that I looked at the harddrive cage to see about the condition of the fan on there....only to find that there was no fan there.

Reviewing the manual on the Gateway website, I found that the cage is used in more than one model...and that some of those models have fans, while evidently mine did not. Browsing the parts list for some of the other Gateway models that used the same drive cage, I found reference to a 60mm x 10mm fan, which I deduced was probably the fan that I would need to get for this location. I did find that from the Gateway manual for my model, that the motherboard did have a front chassis fan connector.

So, after some thought and checking first amazon.com, it struck me that eBay might be the better place to go. So I found a seller on eBay that explicitly said he shipped by USPS and bought one, and from another seller I got fan screws (a bunch of them, because I've needed them in the past before and I'm sure I'll have need for them in the future....as it was I didn't actually need them this time though.)

Because I had recently built my new backuppc pool (should be posting about that adventure some day), I had been waiting for a moment when Orac was idle again and not busy refilling the pool with full backups of everything....It hadn't gotten any fulls of Zen yet, it didn't detect that Zen had gone away to apply the recent Microsoft patches during its recent attempt, so I had to step in and stop it. So that seemed like a good time to take Orac down.

Pages: 1· 2· 3· 4

Now instead of subjecting some poor random forum to a long rambling thought, I will try to consolidate those things into this blog where they can be more easily ignored profess to be collected thoughts from my mind.

Latest Poopli Updaters -- http://lkc.me/poop

bloglovin

There are 20 years 1 month 25 days 21 hours 42 minutes and 45 seconds until the end of time.
And, it has been 4 years 11 months 1 day 16 hours 20 minutes and 11 seconds since The Doctor saved us all from the end of the World!

Search

November 2017
Mon Tue Wed Thu Fri Sat Sun
 << <   > >>
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30      
Google

Linkblog

  XML Feeds

Who's Online?

  • Guest Users: 0
This seal is issued to lawrencechen.net by StopTheHacker Inc.
powered by b2evolution free blog software

hosted by
Green Web Hosting! This site hosted by DreamHost.

monitored by
Monitored by eXternalTest
SiteUptime Web Site Monitoring Service
website uptime