Hi,

i have the problem with creating new instances from templates. Using KVM on Centos 6.2 as hypervisors. Primary storage is SharedPoint type. I can create new instance from scratch to boot from ISO. When i create it from template it failed with this details in log:


2012-02-05 22:42:13,342 WARN [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-5:job-1363) Cleanup failed due to org.libvirt.LibvirtException: Storage volume not found: no storage vol with matching key for com.cloud.agent.api.Answer
2012-02-05 22:42:13,342 INFO [cloud.storage.StorageManagerImpl] (Job-Executor-5:job-1363) Will retry delete of Vol[257|vm=219|ROOT] from 200
2012-02-05 22:42:13,343 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-5:job-1363) Looking for pools in dc: 1 pod:1 cluster:1
2012-02-05 22:42:13,344 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-5:job-1363) FirstFitStoragePoolAllocator has 1 pools to check for allocation
2012-02-05 22:42:13,344 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-5:job-1363) Checking if storage pool is suitable, name: c01 ,poolId: 200
2012-02-05 22:42:13,344 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-5:job-1363) StoragePool is in avoid set, skipping this pool
2012-02-05 22:42:13,344 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-5:job-1363) FirstFitStoragePoolAllocator returning 0 suitable storage pools
2012-02-05 22:42:13,344 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-5:job-1363) No suitable pools found for volume: Vol[259|vm=247|ROOT] under cluster: 1
2012-02-05 22:42:13,344 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-5:job-1363) No suitable pools found
2012-02-05 22:42:13,344 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-5:job-1363) No suitable storagePools found under this Cluster: 1
2012-02-05 22:42:13,344 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-5:job-1363) Could not find suitable Deployment Destination for this VM under any clusters, returning.
2012-02-05 22:42:13,346 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-5:job-1363) VM state transitted from :Starting to Stopped with event: OperationFailedvm's original host id: null new host id: null host id before state transition: 38
2012-02-05 22:42:13,348 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-5:job-1363) Hosts's actual total CPU: 63816 and CPU after applying overprovisioning: 63816
2012-02-05 22:42:13,348 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-5:job-1363) release cpu from host: 38, old used: 53700,reserved: 0, actual total: 63816, total with overprovisioning: 63816; new used: 500,reserved:0; movedfromreserved: false,moveToReserveredfalse
2012-02-05 22:42:13,349 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-5:job-1363) release mem from host: 38, old used: 21239955456,reserved: 0, total: 25180418048; new used: 268435456,reserved:0; movedfromreserved: false,moveToReserveredfalse
2012-02-05 22:42:13,353 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-5:job-1363) VM state transitted from :Stopped to Error with event: OperationFailedToErrorvm's original host id: null new host id: null host id before state transition: null
2012-02-05 22:42:13,365 INFO [api.commands.DeployVMCmd] (Job-Executor-5:job-1363) com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-247-VM]Scope=interface com.cloud.dc.DataCenter; id=1
2012-02-05 22:42:13,365 WARN [cloud.api.ApiDispatcher] (Job-Executor-5:job-1363) class com.cloud.api.ServerApiException : Unable to create a deployment for VM[User|i-2-247-VM]
2012-02-05 22:42:13,366 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-5:job-1363) Complete async job-1363, jobStatus: 2, resultCode: 530, result: com.cloud.api.response.ExceptionResponse@40aca68a

Thank you for the help.
Michal

Hi Michal,
Can you please check if the storage pool has sufficient allocated capacity (look into the details of this storage pool) and is in the correct status ? Are there any logs/exceptions before this snippet ?

Hi,

actually the pool has 5.5TB and 3.76 is used. CloudStack has allocated now 100GB (info from dashboard). From that point i guess it is not the free space problem. Status of the pool is UP.

Today i tried again to create an instance from template to get log again. And suddenly it works but when i tried to create another one it does not work again. And in the log i am not able to find any exact exception just this.

012-02-06 10:19:06,135 WARN [cloud.vm.VirtualMachineManagerImpl] (AgentManager-Handler-8:null) Cleanup failed due to org.libvirt.LibvirtException: Storage volume not found: no storage vol with matching key for com.cloud.agent.api.Answer
2012-02-06 10:19:06,135 DEBUG [agent.transport.Request] (Job-Executor-12:job-1370) Seq 37-678757192: Received: { Ans: , MgmtId: 116529676052, via: 37, Ver: v1, Flags: 110, { Answer } }
2012-02-06 10:19:06,135 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-8:null) Seq 37-678757192: No more commands found
2012-02-06 10:19:06,135 WARN [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-12:job-1370) Cleanup failed due to org.libvirt.LibvirtException: Storage volume not found: no storage vol with matching key for com.cloud.agent.api.Answer
2012-02-06 10:19:06,136 INFO [cloud.storage.StorageManagerImpl] (Job-Executor-12:job-1370) Will retry delete of Vol[257|vm=219|ROOT] from 200
2012-02-06 10:19:06,136 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-12:job-1370) Looking for pools in dc: 1 pod:1 cluster:1
2012-02-06 10:19:06,137 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-12:job-1370) FirstFitStoragePoolAllocator has 1 pools to check for allocation
2012-02-06 10:19:06,137 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-12:job-1370) Checking if storage pool is suitable, name: c01 ,poolId: 200
2012-02-06 10:19:06,137 DEBUG [storage.allocator.AbstractStoragePoolAllocator] (Job-Executor-12:job-1370) StoragePool is in avoid set, skipping this pool
2012-02-06 10:19:06,137 DEBUG [storage.allocator.FirstFitStoragePoolAllocator] (Job-Executor-12:job-1370) FirstFitStoragePoolAllocator returning 0 suitable storage pools
2012-02-06 10:19:06,137 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-12:job-1370) No suitable pools found for volume: Vol[262|vm=250|ROOT] under cluster: 1
2012-02-06 10:19:06,137

Well and on the hypervisor i can see this in agent log:

libvir: QEMU error : Domain not found: no domain with matching uuid 'f257919b-0e2e-3d85-823e-c36d79e10727'
libvir: Storage error : Storage volume not found: no storage vol with matching key
libvir: Storage error : invalid argument: storagePoolRefresh: unsupported flags (0xf87ead48)
libvir: Storage error : Storage volume not found: no storage vol with matching key
libvir: Storage error : Storage volume not found: no storage vol with matching key
libvir: Storage error : invalid argument: storagePoolRefresh: unsupported flags (0x8d960bb4)
libvir: Storage error : Storage volume not found: no storage vol with matching key
libvir: Storage error : Storage volume not found: no storage vol with matching key
libvir: Storage error : invalid argument: storagePoolRefresh: unsupported flags (0x8e06bb34)
libvir: Storage error : Storage volume not found: no storage vol with matching key
libvir: Storage error : Storage volume not found: no storage vol with matching key
libvir: Storage error : invalid argument: storagePoolRefresh: unsupported flags (0xec1bcc6c)
libvir: Storage error : Storage volume not found: no storage vol with matching key
libvir: Storage error : Storage volume not found: no storage vol with matching key
libvir: Storage error : invalid argument: storagePoolRefresh: unsupported flags (0x8d960bb4)
libvir: Storage error : Storage volume not found: no storage vol with matching key

Hi,

I got a same problelm..
Had this problem already been fixed?

Thanks,

Unfortunately not ;o(